-- Logs begin at Fri 2022-10-28 08:59:46 UTC, end at Fri 2022-10-28 10:09:16 UTC. -- Oct 28 09:29:51.170202 dv-cinder-zed-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Oct 28 09:29:51.180161 dv-cinder-zed-3-smb devstack@keystone.service[81599]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 28 09:29:51 2022] *** Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: nodename: dv-cinder-zed-3-smb Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: machine: x86_64 Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: clock source: unix Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: pcre jit disabled Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: detected number of CPU cores: 4 Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: current working directory: / Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: detected binary path: /usr/bin/uwsgi-core Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: your processes number limit is 47680 Oct 28 09:29:51.182176 dv-cinder-zed-3-smb devstack@keystone.service[81599]: your memory page size is 4096 bytes Oct 28 09:29:51.182818 dv-cinder-zed-3-smb devstack@keystone.service[81599]: detected max file descriptor number: 2048 Oct 28 09:29:51.182818 dv-cinder-zed-3-smb devstack@keystone.service[81599]: lock engine: pthread robust mutexes Oct 28 09:29:51.182818 dv-cinder-zed-3-smb devstack@keystone.service[81599]: thunder lock: enabled Oct 28 09:29:51.182818 dv-cinder-zed-3-smb devstack@keystone.service[81599]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 28 09:29:51.182818 dv-cinder-zed-3-smb devstack@keystone.service[81599]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 28 09:29:51.221939 dv-cinder-zed-3-smb devstack@keystone.service[81599]: Python main interpreter initialized at 0x55bd9b5559c0 Oct 28 09:29:51.221939 dv-cinder-zed-3-smb devstack@keystone.service[81599]: python threads support enabled Oct 28 09:29:51.221939 dv-cinder-zed-3-smb devstack@keystone.service[81599]: your server socket listen backlog is limited to 100 connections Oct 28 09:29:51.221939 dv-cinder-zed-3-smb devstack@keystone.service[81599]: your mercy for graceful operations on workers is 90 seconds Oct 28 09:29:51.222298 dv-cinder-zed-3-smb devstack@keystone.service[81599]: mapped 403077 bytes (393 KB) for 2 cores Oct 28 09:29:51.222373 dv-cinder-zed-3-smb devstack@keystone.service[81599]: *** Operational MODE: preforking *** Oct 28 09:29:51.222627 dv-cinder-zed-3-smb devstack@keystone.service[81599]: *** uWSGI is running in multiple interpreter mode *** Oct 28 09:29:51.222627 dv-cinder-zed-3-smb devstack@keystone.service[81599]: spawned uWSGI master process (pid: 81599) Oct 28 09:29:51.222831 dv-cinder-zed-3-smb systemd[1]: Started Devstack devstack@keystone.service. Oct 28 09:29:51.223951 dv-cinder-zed-3-smb devstack@keystone.service[81599]: spawned uWSGI worker 1 (pid: 81610, cores: 1) Oct 28 09:29:51.223951 dv-cinder-zed-3-smb devstack@keystone.service[81599]: spawned uWSGI worker 2 (pid: 81611, cores: 1) Oct 28 09:29:51.224514 dv-cinder-zed-3-smb devstack@keystone.service[81599]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 28 09:29:55.527721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:29:55.527721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Oct 28 09:29:55.528575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:29:55.528575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Oct 28 09:29:55.528904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:29:55.528904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: from cryptography.utils import int_from_bytes Oct 28 09:29:55.529353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 28 09:29:55.529353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: from cryptography.utils import int_from_bytes Oct 28 09:29:55.579221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 09:29:55.579425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 09:29:55.579583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] command line args: [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 09:29:55.579854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 28 09:29:55.579939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 09:29:55.580051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ================================================================================ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 09:29:55.580213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 28 09:29:55.580213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] command line args: [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 28 09:29:55.580411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.580500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 28 09:29:55.580572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] config_dir = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.580688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ================================================================================ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 28 09:29:55.580776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.580858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] config_source = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.580966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] admin_token = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] debug = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] config_dir = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] config_source = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] debug = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.581980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_config_append = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.582992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_dir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_config_append = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_dir = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.583934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.584942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.585845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.586985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] publish_errors = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] publish_errors = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.587984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] standard_threads = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.588907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] standard_threads = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] use_journal = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] use_json = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] use_stderr = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] use_journal = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.589921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] use_syslog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.590034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] use_json = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.590160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.590238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] use_stderr = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.590238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] use_syslog = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.590362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.590431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 28 09:29:55.590538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.590947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.591958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.592928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.593970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.594959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.595915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.596867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.597955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.598961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.599968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.600935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.601933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.602984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.603899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.604983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.605914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.606972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.607896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.608982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.609926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.610946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.611934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.612878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.613945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.614973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.615145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.615230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.615384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.615463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.615629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.615705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.616413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.external = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.616524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.616646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.616741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.616903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.external = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.616984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.password = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.token = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.617940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.password = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] auth.token = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.618852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.619975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.620938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.621907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.622988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.623994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.624923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.625896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.626845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.627968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.628910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.629946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.630903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.631062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.631141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.631292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.631370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.632948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.633903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.634979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.635931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.636878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.637979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.638924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.639979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.640937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.641857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.642965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.643940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.644966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.645923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.646856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.647883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.648967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.649993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] role.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] role.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.650925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] role.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] role.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.651917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.652895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653047 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.653850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.654922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.655919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.656964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.657885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.658992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.659964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.660916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.661939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.662941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.663949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.664900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.665980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.666926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.667005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.667153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.667234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.667391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.667469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.667921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.668005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.668005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 09:29:55.668155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.668155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 28 09:29:55.668284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81610) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 28 09:29:55.697959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.698192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.698419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.698846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.699081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.699306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.706840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.707710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.735721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.735966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.736512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.736748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.790372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.790531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.838755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.838982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.839690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.839954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.996432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.996668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.996763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.996878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.996989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:55.997747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.020418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.020650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.020771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.020875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.021893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.022138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.032877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.033197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.046768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.046968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.047876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.048014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.048097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.048289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81610) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 28 09:29:56.065944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.066142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 09:29:56.066268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.066485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 09:29:56.066569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.066674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 09:29:56.066834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.066943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.067025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 28 09:29:56.067206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.067297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.067467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.067548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.067728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.067840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.068079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.078018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.078314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.078398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.078586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.078693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.078872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.079008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.079237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.086498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.086805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.089442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.089727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.089813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.090014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.090097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.090296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.090394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.090832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.090922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.091924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.092020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.092193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.092276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.092463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.092561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.092731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.115491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.115701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.117020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.117213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.117298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.117500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.121955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.122124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.122244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.122415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.122513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.122695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.128853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.128985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.129145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.129258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.135164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.135309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.135465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.135553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.135767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.135875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.136986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.137174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.137256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.153071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.153172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.153359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.153460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.157169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.157262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.160380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.160467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.160681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.160766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.165376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.165502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.168466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.168624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.171630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.171795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.171970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.172942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.173105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.173219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.188867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.189028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.189161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.189322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.189434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.189593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.189709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.189852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.196483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.196758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.197992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.198303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.199491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.199792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.199929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.200131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.202443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.202732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.202815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.203021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.203106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.203304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.203388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.203587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.203700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.204009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.215338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.215903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.216012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.216188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.216272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.216470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.216554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.216797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.216881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.217122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.217205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.217411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.217494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.217708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.233255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.233597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.236152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.236562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.239025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.239527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.240534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.241061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.242019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.242305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.242530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.242697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.242850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.243222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.249110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.249585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.250565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.251057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.253471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.253866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.253966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.254153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.254330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.254414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.254619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.254899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.262221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.262521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.262771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.262856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.263067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.263352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.269659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.269944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.270229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.270313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81610) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 28 09:29:56.270512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.270605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.270781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.270865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.271902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.272492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.272728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.272802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.273986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.274123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.274394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81610) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 28 09:29:56.308742 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bd9b5559c0 pid: 81611 (default app) Oct 28 09:29:56.309172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bd9b5559c0 pid: 81610 (default app) Oct 28 09:29:56.312098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e727d64-6f0c-4cb8-82e7-3300c7f3592c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:29:56.312381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e727d64-6f0c-4cb8-82e7-3300c7f3592c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:29:56.312650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e727d64-6f0c-4cb8-82e7-3300c7f3592c None None] PATH_INFO: `/v3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:29:56.316778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Fri Oct 28 09:29:56 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:03.249393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:03.249916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:03.249916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:03.254187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:03.260604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:03.261169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:03.261169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:03.331283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: INFO dbcounter [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] Registered counter for database keystone Oct 28 09:30:03.351380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG oslo_db.sqlalchemy.engines [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81611) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:30:03.357741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writer thread running {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:30:03.536715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81611) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 28 09:30:03.537107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81611) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 28 09:30:03.596426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:03.598032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:03.729381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 28 09:30:04.197951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:04.267331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: INFO dbcounter [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] Registered counter for database keystone Oct 28 09:30:04.289250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG oslo_db.sqlalchemy.engines [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81610) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 28 09:30:04.295470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writer thread running {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 28 09:30:04.413516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744329b4-0fc9-40fe-bea7-d605eb7c2ad2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:04.417625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:04.418241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:04.418515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:04.418779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:04.426889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:04.480205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 28 09:30:04.480535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1132181-e595-439b-b4ac-90addb2b10f9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:04.491526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Fri Oct 28 09:30:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 09:30:04.497903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:04.516255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba22b33-7fbd-4ca2-a4cc-b6636a294289 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:04.520868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:04.522130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:04.522410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:04.522682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] PATH_INFO: `/v3/projects/afd3325807f94fa3b6a7d418f71aaa51` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:04.524356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] RBAC: Authorizing `identity:get_project(project_id=afd3325807f94fa3b6a7d418f71aaa51)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:04.577573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING keystone.common.rbac_enforcer.enforcer [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 28 09:30:04.577925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d6b40c-cc06-4145-920d-33ca91e64e20 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:04.582593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Fri Oct 28 09:30:04 2022] GET /identity/v3/projects/afd3325807f94fa3b6a7d418f71aaa51 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:06.725757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710458-38db-4f33-8a79-7e4eb2e76e36 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:06.726301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710458-38db-4f33-8a79-7e4eb2e76e36 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:06.726403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9710458-38db-4f33-8a79-7e4eb2e76e36 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:06.730779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:06.736510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:06.736932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:06.737051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:06.756582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:06.758065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:06.765745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 28 09:30:07.033531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9710458-38db-4f33-8a79-7e4eb2e76e36 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:07.045631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9710458-38db-4f33-8a79-7e4eb2e76e36 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:07.049878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa9ea6-8841-4366-b440-fb3f4d14ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:07.050480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa9ea6-8841-4366-b440-fb3f4d14ed33 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:07.050759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa9ea6-8841-4366-b440-fb3f4d14ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:07.051031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa9ea6-8841-4366-b440-fb3f4d14ed33 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:07.058646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa9ea6-8841-4366-b440-fb3f4d14ed33 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:07.060456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa9ea6-8841-4366-b440-fb3f4d14ed33 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:07.069933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Fri Oct 28 09:30:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:30:07.075610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:07.085335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57909c91-4cf5-4d02-b679-6b42cc9ebc67 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:07.090091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093490a3-6a12-45f6-a5dc-b14d92bb8296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:07.090715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093490a3-6a12-45f6-a5dc-b14d92bb8296 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:07.090990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093490a3-6a12-45f6-a5dc-b14d92bb8296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:07.091255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093490a3-6a12-45f6-a5dc-b14d92bb8296 admin admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:07.092560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093490a3-6a12-45f6-a5dc-b14d92bb8296 admin admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:07.094386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093490a3-6a12-45f6-a5dc-b14d92bb8296 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:07.095934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Fri Oct 28 09:30:07 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:09.323790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f47bd-a148-4f3e-9842-bb8a59ba1bdf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:09.324322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f47bd-a148-4f3e-9842-bb8a59ba1bdf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:09.324322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f47bd-a148-4f3e-9842-bb8a59ba1bdf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:09.325652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:09.330953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:09.331420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:09.331617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:09.350195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:09.351356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:09.365624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a3982-75a7-42bf-a7c4-caa7bc5edd7e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:09.365904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a3982-75a7-42bf-a7c4-caa7bc5edd7e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:09.366182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a3982-75a7-42bf-a7c4-caa7bc5edd7e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:09.367543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:30:09.372999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:09.373276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:09.373550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:09.416094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG passlib.handlers.bcrypt [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81610) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 28 09:30:09.416411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG passlib.handlers.bcrypt [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81610) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 28 09:30:09.434430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:09.468833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:09.471428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:09.573002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:09.706154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:09.733167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:09.819077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1486a-e38c-4544-89a5-40a8dc48f5af None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:09.841452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:09.857538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:09.858186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:09.858186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:09.858974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:09.871521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:09.881148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:09.882334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:09.883331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:09.890349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:09.968263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a702b1-79a6-4b3d-b88c-c893919806bd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:09.970443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:09.979280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:09.998454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b7b69-0bac-48ad-8d36-7c8f3d802ed7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:10.007575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.008474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.008753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.009022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.033612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.034715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.035184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.035641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.037183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:10.037616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.039142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.051062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:10.057526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:10.080516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.082064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.185048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e8cd0-8b34-4b23-be41-dba596e3e5b1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:10.203910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:30:10.209529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:10.213482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:10.221486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.222049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.222325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.222593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.224466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.225959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.227701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:10 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:10.233427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:10.251802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:10.321453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74dc39-4abb-4c9b-812b-e84f973739d3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:10.340574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08078073-bd3c-45c8-a0ac-e2dea7dcc3b0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:10.360145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.361018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.361296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.361571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.377640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.378288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.378570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.378848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.380017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.381302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.381474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.382753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.383050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:10.388892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:30:10.401976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:10.403097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:30:10.404125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:30:10.404125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:30:10.404125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:10.404125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:10.409102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:10.468375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:10.492570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:10.507079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26abdb9-90ad-4d1a-bc91-0a77454c8417 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:10.518169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb620132-890a-4f27-8712-e4e4d71ee99d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:10.545564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.546178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.546451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.546717 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.547437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.domain.id=default, user.id=66bbceeaa7394d2b922231b2c11b4f8f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:10.547758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.549221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a6cdb-6e12-440d-ae91-5c5f6e0d2057 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.556543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Fri Oct 28 09:30:10 2022] GET /identity/v3/role_assignments?user.id=66bbceeaa7394d2b922231b2c11b4f8f&scope.domain.id=default&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 233 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:10.556747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:10.557260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:10.557537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:10.557807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:10.558457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:10.558759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:10.560228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bf675-f1c2-407e-b0bf-996f66aa2484 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:10.580979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:13.189533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba475-c266-47b6-a211-b42d056495f7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:13.189533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba475-c266-47b6-a211-b42d056495f7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:13.189533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ba475-c266-47b6-a211-b42d056495f7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:13.189533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:13.196219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:13.197126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:13.197594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:13.229380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:13.230893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:13.371672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:13.867735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:13.867969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64c3e6-b887-4e21-8250-6fd28768e7ce None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:13.868052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64c3e6-b887-4e21-8250-6fd28768e7ce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:13.868343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64c3e6-b887-4e21-8250-6fd28768e7ce None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:13.869636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 13/24] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:13.881438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae6791-d910-4ea3-95e6-20f5d1753465 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:13.881751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae6791-d910-4ea3-95e6-20f5d1753465 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:13.881991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae6791-d910-4ea3-95e6-20f5d1753465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:13.920938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-abae6791-d910-4ea3-95e6-20f5d1753465 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:13.922188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-abae6791-d910-4ea3-95e6-20f5d1753465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:13.930079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:14.100336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd146120-ae47-40fa-95b5-f6314ab8895b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:14.154626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:14.162592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b517ee5-2603-42cf-9bea-21d4ea51ced2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.163116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b517ee5-2603-42cf-9bea-21d4ea51ced2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.163542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b517ee5-2603-42cf-9bea-21d4ea51ced2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.165841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 15/26] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:30:14.168870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:14.169808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.170317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.170795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.182874 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153a52-605d-4964-8a6b-5b161de5f427 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.183490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153a52-605d-4964-8a6b-5b161de5f427 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.185598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35153a52-605d-4964-8a6b-5b161de5f427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.214000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-35153a52-605d-4964-8a6b-5b161de5f427 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:14.216299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35153a52-605d-4964-8a6b-5b161de5f427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:14.232359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:14.235622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:14.236510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:14.237285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:14.244208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 12/27] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:14.251110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:14.373851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 16/28] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:14.503816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af2a88-bebf-45d5-9dc6-fc48dfe0f9cb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.505031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af2a88-bebf-45d5-9dc6-fc48dfe0f9cb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.505512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af2a88-bebf-45d5-9dc6-fc48dfe0f9cb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.506983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 17/29] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:30:14.514452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13627d-1642-4e1f-9b7c-42ac4bb089e5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:14.521756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.522316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.522807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.524479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:14.542498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:14.554043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:14.554663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.554939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.555213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.556667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:14.556974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:14.558366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:14.581043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 13/30] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:14.583609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:14.584922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:14.587361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:14.767464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 18/31] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:14.772356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:14.786483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efa40-223f-4a0d-b1b6-b61d83fbb0a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:14.859653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:14.860372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:14.860648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:14.861124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:14.864071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:14.865546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:14.871230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 14/32] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:14 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:14.876909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:14.928173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:14.998146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde7e76-95f0-4929-bd31-6a0a0d442b51 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.067378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.068379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.068878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.069350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.070249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.071976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.073034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcac3e-4fdc-49d3-8b47-0c45e2eaecfc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.079623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:15.118152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.118815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.119096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.119420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.137697 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.139261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:30:15.158289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:15.159151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:30:15.165744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:30:15.165744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:30:15.165744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:15.165744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 15/33] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:15.165744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:15.229326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 19/34] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Oct 28 09:30:14 2022] POST /identity/v3/roles => generated 219 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:15.231202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:15.257490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:15.279785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bef0f2-3eed-42fb-b536-39c50b820f31 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.314253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.316085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.316085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.316085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.317787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.319260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.321722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 16/35] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:15.321988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:15.453977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9f349-71f2-438a-bafc-6cc6fece7a0d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.458123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdc350-18a7-436b-855f-eebfef42ad24 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.500151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.500894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.501178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.501455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.501955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31bff8-9c92-480d-8b14-b7a8adfbe03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.502184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:15.502442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.503213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31bff8-9c92-480d-8b14-b7a8adfbe03c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.503703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31bff8-9c92-480d-8b14-b7a8adfbe03c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.503886 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.504916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31bff8-9c92-480d-8b14-b7a8adfbe03c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.506120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31bff8-9c92-480d-8b14-b7a8adfbe03c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.507944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31bff8-9c92-480d-8b14-b7a8adfbe03c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.517320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 17/36] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:15.518186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:15.531784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 20/37] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Oct 28 09:30:15 2022] POST /identity/v3/roles => generated 225 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:15.536770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a9afc-3d75-41e5-85b4-c3422d274540 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.537051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a9afc-3d75-41e5-85b4-c3422d274540 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.537332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a9afc-3d75-41e5-85b4-c3422d274540 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.538740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 21/38] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:15.540884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3942eb-1627-47e3-b35f-abe7d7503c5b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.540884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3942eb-1627-47e3-b35f-abe7d7503c5b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.540884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3942eb-1627-47e3-b35f-abe7d7503c5b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.542620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 22/39] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:15.546969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3942eb-1627-47e3-b35f-abe7d7503c5b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:15.550802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:15.637462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:15.668590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611da983-23c1-4c42-9b73-0e2adf719daa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.699860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3942eb-1627-47e3-b35f-abe7d7503c5b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:15.711280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301b7aa-131d-4026-91ed-c484000924d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.712089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301b7aa-131d-4026-91ed-c484000924d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.712369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301b7aa-131d-4026-91ed-c484000924d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.713295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301b7aa-131d-4026-91ed-c484000924d6 None admin] PATH_INFO: `/v3/domains/default/users/66bbceeaa7394d2b922231b2c11b4f8f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.715712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301b7aa-131d-4026-91ed-c484000924d6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=66bbceeaa7394d2b922231b2c11b4f8f, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.717595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301b7aa-131d-4026-91ed-c484000924d6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.748360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58dabdd-694e-4f5b-afcd-c354231b878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:15.748983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dabdd-694e-4f5b-afcd-c354231b878f None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.749286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dabdd-694e-4f5b-afcd-c354231b878f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.749558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dabdd-694e-4f5b-afcd-c354231b878f None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.750273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58dabdd-694e-4f5b-afcd-c354231b878f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:15.751725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58dabdd-694e-4f5b-afcd-c354231b878f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:15.787532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 23/40] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:30:15 2022] POST /identity/v3/projects => generated 301 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:15.789850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.789850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.789850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.809992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:15.811564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:15.843285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 18/41] 10.222.0.52 () {58 vars in 1361 bytes} [Fri Oct 28 09:30:15 2022] PUT /identity/v3/domains/default/users/66bbceeaa7394d2b922231b2c11b4f8f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:15.845095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:15.845386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:15.845661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:15.872446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:15.874221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:15.912782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 24/42] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:15.968871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 19/43] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:16.186962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:16.226174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:16.252739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:16.262410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:16.373827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566ac07-9d6e-4246-96b9-0229986ebdcb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:16.406425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d702c-cdcf-4821-8690-76c89aaff755 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:16.416202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:16.417172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:16.417660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:16.418010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:16.419189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:16.421013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:16.426573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 20/44] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:16.432340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:16.449517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d6406-75cb-4342-8716-5d91d6c52c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:16.450460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d6406-75cb-4342-8716-5d91d6c52c06 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:16.450896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d6406-75cb-4342-8716-5d91d6c52c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:16.451182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d6406-75cb-4342-8716-5d91d6c52c06 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:16.451911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d6406-75cb-4342-8716-5d91d6c52c06 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:16.453312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d6406-75cb-4342-8716-5d91d6c52c06 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:16.472313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 25/45] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Oct 28 09:30:16 2022] POST /identity/v3/roles => generated 223 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:16.525805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:16.556908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914ba8e-c038-4a1d-afb6-bebcdb14b37f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:16.605658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4746dd-1d3e-40eb-85b3-21d9988ea526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:16.606294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4746dd-1d3e-40eb-85b3-21d9988ea526 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:16.606569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4746dd-1d3e-40eb-85b3-21d9988ea526 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:16.606840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4746dd-1d3e-40eb-85b3-21d9988ea526 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:16.607529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4746dd-1d3e-40eb-85b3-21d9988ea526 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:16.609213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4746dd-1d3e-40eb-85b3-21d9988ea526 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:16.754765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 21/46] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:30:16 2022] POST /identity/v3/projects => generated 312 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:30:18.055755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c06038-244e-4a4f-887a-43100917f026 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.056271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c06038-244e-4a4f-887a-43100917f026 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.056353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c06038-244e-4a4f-887a-43100917f026 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.057650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 26/47] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:18.063058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.063534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.063616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.081920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:18.083092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:18.153675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 22/48] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:18.421656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c06038-244e-4a4f-887a-43100917f026 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:18.521655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:18.529634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c06038-244e-4a4f-887a-43100917f026 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:18.566683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:18.567322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.567600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.567908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.577961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:18.583441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:18.584469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:18.585258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 27/49] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:18.590658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:18.649396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:18.693621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc345e87-8e36-4472-b875-e67e1004dbfc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:18.727951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:18.728723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.729004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.729277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.729945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:18.730246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:18.731640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:18.740029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 23/50] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:18.746225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:18.850665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42a031-4a27-40f4-beef-bc933595dbb3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:18.886554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a0575-1fe2-4dc3-b799-2381b06c2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:18.887191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0575-1fe2-4dc3-b799-2381b06c2d78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.887470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0575-1fe2-4dc3-b799-2381b06c2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.887758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0575-1fe2-4dc3-b799-2381b06c2d78 None admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.888889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a0575-1fe2-4dc3-b799-2381b06c2d78 None admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:18.890354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a0575-1fe2-4dc3-b799-2381b06c2d78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:18.891968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 28/51] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:18 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:18.897906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:18.921837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955ce1-703a-4ae8-bc00-ac3a2b1cdd3a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.922123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955ce1-703a-4ae8-bc00-ac3a2b1cdd3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.922397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54955ce1-703a-4ae8-bc00-ac3a2b1cdd3a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.923713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 29/52] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:30:18.928936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:18.929223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:18.929501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:18.947878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:18.949018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:18.971451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:19.009923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ce390-9372-4b86-8007-c389d981209d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:19.027039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 30/53] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:19.031224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:19.048635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:19.049274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:19.049553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:19.049825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:19.051002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:19.052487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:19.054072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 24/54] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:19.061140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:19.177792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab5eb0-006f-4e9e-a2c6-3bc6c401cc02 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:19.216931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:19.217555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:19.217904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:19.218275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:19.219103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.domain.id=default, user.id=66bbceeaa7394d2b922231b2c11b4f8f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:19.219472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:19.221036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:19.227151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 31/55] 10.222.0.52 () {56 vars in 1322 bytes} [Fri Oct 28 09:30:19 2022] GET /identity/v3/role_assignments?user.id=66bbceeaa7394d2b922231b2c11b4f8f&scope.domain.id=default&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 530 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:19.299280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:19.418827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1841a2-4e5e-4fc2-b06e-c3940b17bceb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:19.464759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cca5c0-0ad3-4c05-8f8d-2f2702193b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:19.465408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca5c0-0ad3-4c05-8f8d-2f2702193b1e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:19.465689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca5c0-0ad3-4c05-8f8d-2f2702193b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:19.465962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca5c0-0ad3-4c05-8f8d-2f2702193b1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:19.467143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cca5c0-0ad3-4c05-8f8d-2f2702193b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:19.468571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cca5c0-0ad3-4c05-8f8d-2f2702193b1e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:19.470137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 25/56] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:19.477162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:19.496838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:19.576360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6120e6-d6ba-4450-9410-bfe4e68d2b8b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:19.608312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90854144-68df-4e2a-896f-11d85d2bf164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:19.608920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854144-68df-4e2a-896f-11d85d2bf164 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:19.609304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854144-68df-4e2a-896f-11d85d2bf164 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:19.609464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90854144-68df-4e2a-896f-11d85d2bf164 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:19.610169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90854144-68df-4e2a-896f-11d85d2bf164 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:19.611574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90854144-68df-4e2a-896f-11d85d2bf164 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:19.639931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 32/57] 10.222.0.52 () {60 vars in 1140 bytes} [Fri Oct 28 09:30:19 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:21.835262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951e7b0-b8a6-48c3-b3e8-d19d04a105c1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:21.835797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951e7b0-b8a6-48c3-b3e8-d19d04a105c1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:21.835888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951e7b0-b8a6-48c3-b3e8-d19d04a105c1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:21.837155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 26/58] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:21.843291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:21.843723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:21.843878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:21.868645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:21.869778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:21.938977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:21.945808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 33/59] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:22.220596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951e7b0-b8a6-48c3-b3e8-d19d04a105c1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:22.281399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:22.328999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951e7b0-b8a6-48c3-b3e8-d19d04a105c1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:22.365836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1339d48-b8a9-4260-9e08-937d577d036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:22.366484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1339d48-b8a9-4260-9e08-937d577d036b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:22.366789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1339d48-b8a9-4260-9e08-937d577d036b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:22.367066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1339d48-b8a9-4260-9e08-937d577d036b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:22.368217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1339d48-b8a9-4260-9e08-937d577d036b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:22.369700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1339d48-b8a9-4260-9e08-937d577d036b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:22.371315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 27/60] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:22.377031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:22.483483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dc5cd-0656-4d59-86d7-16796189c372 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:22.521301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2555ec9-c877-4b9b-ae81-106bdb6b5ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:22.521910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555ec9-c877-4b9b-ae81-106bdb6b5ad8 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:22.522188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555ec9-c877-4b9b-ae81-106bdb6b5ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:22.522462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2555ec9-c877-4b9b-ae81-106bdb6b5ad8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:22.523121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555ec9-c877-4b9b-ae81-106bdb6b5ad8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:22.524627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2555ec9-c877-4b9b-ae81-106bdb6b5ad8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:22.550767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 34/61] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:30:22 2022] POST /identity/v3/users => generated 275 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:25.920575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b793d8-a67e-41d6-b5d4-2f70c3568535 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:25.920575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b793d8-a67e-41d6-b5d4-2f70c3568535 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:25.920575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b793d8-a67e-41d6-b5d4-2f70c3568535 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:25.921570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 28/62] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:25.936207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e325c28-3b88-4007-a65a-1c408ddcbf51 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:25.937059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e325c28-3b88-4007-a65a-1c408ddcbf51 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:25.937501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e325c28-3b88-4007-a65a-1c408ddcbf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:25.969022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6e325c28-3b88-4007-a65a-1c408ddcbf51 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:25.970510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6e325c28-3b88-4007-a65a-1c408ddcbf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:26.092507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:26.115481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 35/63] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:26.177003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c5d1-cb99-499a-b1a5-bea8460b4791 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.177617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c5d1-cb99-499a-b1a5-bea8460b4791 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.178091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5c5d1-cb99-499a-b1a5-bea8460b4791 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.180031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 29/64] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:26.185573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c7e16-4b85-4f86-83f6-89316ca44120 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.185905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c7e16-4b85-4f86-83f6-89316ca44120 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.186130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c7e16-4b85-4f86-83f6-89316ca44120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.213122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5e6c7e16-4b85-4f86-83f6-89316ca44120 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:26.214622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e6c7e16-4b85-4f86-83f6-89316ca44120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:26.393221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 36/65] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:26.445021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5c5d1-cb99-499a-b1a5-bea8460b4791 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:26.448365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367f4f6-daa9-4f1d-bfa5-01d621f08cbc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.449824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367f4f6-daa9-4f1d-bfa5-01d621f08cbc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.450307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367f4f6-daa9-4f1d-bfa5-01d621f08cbc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.452135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 37/66] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:26.458349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679b7e-6d73-4ed8-8fef-8b4864310ab9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.458971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679b7e-6d73-4ed8-8fef-8b4864310ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.459638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa679b7e-6d73-4ed8-8fef-8b4864310ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.483282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aa679b7e-6d73-4ed8-8fef-8b4864310ab9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:26.484960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa679b7e-6d73-4ed8-8fef-8b4864310ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:26.560333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 38/67] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:26.562286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642486-dc55-4941-b9c8-a4b558faf76d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.562690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5c5d1-cb99-499a-b1a5-bea8460b4791 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:26.562883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642486-dc55-4941-b9c8-a4b558faf76d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.563250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56642486-dc55-4941-b9c8-a4b558faf76d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.564773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 39/68] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:26.577810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a9537-89a9-4bbd-96ab-4f431ab53e97 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.578449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a9537-89a9-4bbd-96ab-4f431ab53e97 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.579003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a9537-89a9-4bbd-96ab-4f431ab53e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.589851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:26.603293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:26.604284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.604708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.605148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.616812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:26.622081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:26.622925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:26.623714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:26.624842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-436a9537-89a9-4bbd-96ab-4f431ab53e97 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:26.626193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 30/69] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:26.626297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-436a9537-89a9-4bbd-96ab-4f431ab53e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:26.634250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:26.738085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:26.757213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ff100-c74b-4ab4-bd2e-c4a8c758ad23 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:26.796045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:26.797012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.797492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.797959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.798823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 40/70] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:26.799034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:26.799336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:26.801212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d1b3c-fb86-44bb-b82f-58d513abf4af None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.801591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d1909d-4328-4ea6-92cd-596152b2052c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:26.801789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d1b3c-fb86-44bb-b82f-58d513abf4af None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.802217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d1b3c-fb86-44bb-b82f-58d513abf4af None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.803737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 41/71] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:26.811999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d1b3c-fb86-44bb-b82f-58d513abf4af None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:26.818856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 31/72] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:26.825464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.825753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.826028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.844764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:26.845955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:26.888089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:26.923398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 32/73] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:26.931056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d1b3c-fb86-44bb-b82f-58d513abf4af None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:26.933125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:26.970543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:26.971214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:26.971458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:26.971734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:26.984707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:26.991126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:26.992688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:26.994003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:26.996151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 42/74] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:26.997149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.042345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.074717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426c994-3d6d-4ae5-a8ca-1b59cf6b0db1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.117982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdde3ec-1470-425b-82ec-ef26cd55f074 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.131628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.132329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.132609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.132900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.152787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.154355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.157359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 33/75] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:26 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:27.157731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.179809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.180728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.181184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.181655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.193814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:27.198658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:27.200424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:27.201967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:27.202930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:27.204928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 43/76] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:30:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:27.206157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.284861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd41d8-8f14-4ca8-9856-16015d393767 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.323650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.330267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b171e-3892-4bbf-b892-4f8c60109abf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.335644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.336378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.336744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.337093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.337899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:27.338285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.340107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.351163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 34/77] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:27.352069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.375685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.376455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.376752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.377998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.392519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:27.399193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:27.400353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 44/78] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:27.401436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.479983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.487691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf592b6-799c-46cd-881b-9b1ac3f545c9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.520670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8c798-0789-4cf8-b629-84ee8d18f717 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.529266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.529927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.530206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.530478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.564959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.567429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.567851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.568229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.569475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.571018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.572628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 45/79] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:27.573406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.601511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.616748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.618228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.689179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71424591-7aae-44c0-b3c9-2e9a5460c23a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.722132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.729086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.729707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.729989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.730262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.730936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:27.731240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.732539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 35/80] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:27.738271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.738364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.749522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 46/81] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:27.750408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.852159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:27.852323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8d626-92e6-4eec-ad6d-d98993eaec9d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.869085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268ff3c-a11e-4972-bee3-7bac54b062fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:27.891565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.892333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.892686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.893159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.894274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.895839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.897523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 36/82] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:27.898402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.905716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:27.906084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:27.906371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:27.906655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:27.907342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:27.910036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:27.910036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:27.919141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 47/83] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:27.920017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:27.972317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:28.020094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0a3e-89a9-4591-804d-8555f70ef594 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.036205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f79bf-31dc-411b-a5ae-71484dc30e9b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.064606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.065229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.065536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.065852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.066512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.068041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.073342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.073979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.074258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.074532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.075654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.077150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.083789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 48/84] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:27 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:28.083789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:28.097294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 37/85] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:30:27 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:28.098374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:28.105217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:28.232109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdcdc8-6b1b-4375-8da7-c5c24007b2f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.240450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30e439-615e-4ec1-934b-b387d60e5919 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.268164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:28.285623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.285623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.285906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.286170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.287237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.288747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.290352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 49/86] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:28 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:28.290352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:28.297938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.298875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.299221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.299600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.399294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.400882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.434855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:28.467801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5dc65-2bb8-4667-8ae6-91881565bb93 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.519446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.521483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.521778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.522037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.531228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.531228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.531228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 50/87] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:28 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:28.531228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:28.581694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 38/88] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:28.582938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:28.608639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:28.745224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4445364-0dff-4f3e-8434-18bbfbbace09 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.794627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef2037-4c88-42b8-ba75-ae8c0f15e037 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:28.800455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.803774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.803774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.803774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.895773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:28.895773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:28.895773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:28.895773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:28.896148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=7f8be41948d446bead6b1ad550c58d55) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:28.896148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.896443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.906722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 51/89] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:28 2022] GET /identity/v3/role_assignments?user.id=7f8be41948d446bead6b1ad550c58d55&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 259 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:28.906722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:28.919451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:28.929729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:28.951783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:28.974397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:29.163378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae603cc-8d8a-41d3-b0ff-28f59a84c934 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:29.179251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 39/90] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:29.181156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:29.236158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:29.237582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:29.237582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:29.237582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:29.367391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:29.369888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:29.389611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:29.434853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a8253-9659-4c38-bff0-c63b291ef4e7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:29.472130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:29.502509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:29.503806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:29.505482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:29.508186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:29.514221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:29.515662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:29.519683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 40/91] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:29 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:29.520494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:29.570835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 52/92] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:29.570835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:29.729695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0356d62-a16e-48c5-8aa1-f5e95b3d9adb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:29.807774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:29.809263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:29.809771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:29.810279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:29.814446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:29.816835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:29.819003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 41/93] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:29 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:29.820069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:29.895911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554d343-93f7-4e52-b1e3-7b229b21b5ca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:29.931047 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:29.952511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:29.985461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:29.985461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:29.985701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:29.985950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:29.986605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=66bbceeaa7394d2b922231b2c11b4f8f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:29.986926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:29.988985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:30.002794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 53/94] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:29 2022] GET /identity/v3/role_assignments?user.id=66bbceeaa7394d2b922231b2c11b4f8f&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:30.004123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:30.045666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b03ce8-09db-46c9-b9b2-23d47be8ef37 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:30.100495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:30.101528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:30.102010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:30.102534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] PATH_INFO: `/v3/projects/9362c10e8edb48d1878f5e153f269516` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:30.104278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9362c10e8edb48d1878f5e153f269516)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:30.107209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:30.112707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 42/95] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:29 2022] GET /identity/v3/projects/9362c10e8edb48d1878f5e153f269516 => generated 312 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:30.120938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:30.227505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79892702-2a74-4c01-bf7d-3335327cb2c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:30.292730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6132755-2991-4351-887d-e52dcf6bc5e1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:30.317462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:30.322996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:30.322996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:30.322996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:30.322996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] RBAC: Adding query filter params (role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=7f8be41948d446bead6b1ad550c58d55) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:30.322996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:30.322996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94930a-e81c-4e66-b65d-8c851c9c95a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:30.323490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:30.336745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 54/96] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:30 2022] GET /identity/v3/role_assignments?user.id=7f8be41948d446bead6b1ad550c58d55&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:30.341842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:30.341842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:30.347119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:30.347119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:30.347119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=9362c10e8edb48d1878f5e153f269516, user.id=7f8be41948d446bead6b1ad550c58d55) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:30.347119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:30.347119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0e4ea-1f14-4008-8ee5-817b867f9cf8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:30.357740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 43/97] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:30 2022] GET /identity/v3/role_assignments?user.id=7f8be41948d446bead6b1ad550c58d55&scope.project.id=9362c10e8edb48d1878f5e153f269516&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:31.101922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fddbaf-4521-4603-a2af-4233dbe3112d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:31.102289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fddbaf-4521-4603-a2af-4233dbe3112d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:31.102484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fddbaf-4521-4603-a2af-4233dbe3112d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:31.104036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 55/98] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:31.109224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:31.109705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:31.109797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:31.140991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:31.142334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:31.231860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a633b-8c78-40b9-988b-0135efde2a7b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:31.231860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a633b-8c78-40b9-988b-0135efde2a7b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:31.231860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54a633b-8c78-40b9-988b-0135efde2a7b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:31.231860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 56/99] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:30:31.238208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:31.238484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:31.238754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:31.251398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 44/100] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:31.276018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:31.277266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:31.355039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:31.404937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 57/101] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:31.693864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:31.823528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5132c-64a0-4282-b8f4-07857fe8459d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:31.843203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:31.873610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:31.874275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:31.874548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:31.874813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:31.876006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:31.877465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:31.879905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 45/102] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:31.889566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:31.944482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:32.094064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf36f9a-5d5c-484c-89a8-bf2703be74c0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:32.166995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4192c8-ad50-459b-92f1-6aa21c5c4060 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:32.172042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:32.173153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:32.173628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:32.174116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:32.213836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:32.234810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:32.235758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:32.236524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:32.239407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 46/103] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:32.256542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:32.313041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cee96-e72b-4697-aa41-89883f155066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:32.314066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cee96-e72b-4697-aa41-89883f155066 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:32.314547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cee96-e72b-4697-aa41-89883f155066 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:32.315025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cee96-e72b-4697-aa41-89883f155066 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:32.316813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cee96-e72b-4697-aa41-89883f155066 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:32.319659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cee96-e72b-4697-aa41-89883f155066 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:32.381019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:32.416282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 58/104] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:30:31 2022] POST /identity/v3/projects => generated 302 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:32.417856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e66da-4d5e-4492-8eb0-6d676a9c9323 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:32.418343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e66da-4d5e-4492-8eb0-6d676a9c9323 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:32.418841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e66da-4d5e-4492-8eb0-6d676a9c9323 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:32.420352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 59/105] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:32.433980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:32.434657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:32.435133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:32.470321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:32.473208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:32.498198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479e4e3-7978-41b8-81d5-47b9e35ae3f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:32.539291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:32.540305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:32.540934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:32.541412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:32.542308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:32.542821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:32.544505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:32.546295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:32.560695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 47/106] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:32.580023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:32.709316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 60/107] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:32.854417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee2d43-a530-4785-9237-5cbea92fccaf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:32.865747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:32.913970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:32.915116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:32.915616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:32.916118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] PATH_INFO: `/v3/users/a130eade9ce34454bf97655e0f9d4590` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:32.948554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] RBAC: Authorizing `identity:get_user(user_id=a130eade9ce34454bf97655e0f9d4590)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:32.950435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:32.952490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 48/108] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:32 2022] GET /identity/v3/users/a130eade9ce34454bf97655e0f9d4590 => generated 289 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:32.961255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:33.169799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbcc7ed-713c-4b4f-876a-b200331fa842 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:33.230279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:33.231657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:33.233824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:33.234118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:33.234399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:33.296864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:33.382477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:33.384000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:33.515247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764b4e75-9e1e-4fea-b98d-791ecf0bb443 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:33.606727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26e95-b998-452e-b273-66115704f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:33.608158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26e95-b998-452e-b273-66115704f737 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:33.609674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26e95-b998-452e-b273-66115704f737 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:33.610374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c26e95-b998-452e-b273-66115704f737 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:33.641220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c26e95-b998-452e-b273-66115704f737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-69c26e95-b998-452e-b273-66115704f737 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:33.648712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:33.649717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:33.651142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:33.663497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 49/109] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:33.673608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26e95-b998-452e-b273-66115704f737 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:33.680022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 61/110] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:33.689044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:33.757282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:33.833695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:33.986105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c26e95-b998-452e-b273-66115704f737 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:33.989936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d56e-64e5-4ef9-99a7-00e87fb22a36 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:34.098895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:34.099943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.100459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.101210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.106144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:34.108012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:34.110011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 62/111] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:33 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:34.117372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:34.137104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:34.137737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.138012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.138281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.138947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:34.139250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:34.140687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:34.166569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 50/112] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 493 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:34.176900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:34.414754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:34.436728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359287e-8193-4573-9b06-e0956045d358 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:34.469883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67691168-ca95-44a0-800d-ae056ad1a32c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:34.475632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e43b8-0859-4a3a-a40b-4992dc15649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:34.476566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e43b8-0859-4a3a-a40b-4992dc15649a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.477068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e43b8-0859-4a3a-a40b-4992dc15649a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.477606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e43b8-0859-4a3a-a40b-4992dc15649a None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.479159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e43b8-0859-4a3a-a40b-4992dc15649a None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:34.481429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e43b8-0859-4a3a-a40b-4992dc15649a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:34.483333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 51/113] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:34 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:34.485594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0e058-3b2f-4eaf-859f-9394e9ac7548 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.486393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0e058-3b2f-4eaf-859f-9394e9ac7548 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.487086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0e058-3b2f-4eaf-859f-9394e9ac7548 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.488685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 52/114] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:34.490328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca065613-7755-457a-af00-b5d7d5f5e2cc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.490876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca065613-7755-457a-af00-b5d7d5f5e2cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.491352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca065613-7755-457a-af00-b5d7d5f5e2cc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.492989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 53/115] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:34.498819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910b3a-7e7e-43d1-aff8-1fe61f46537c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.499385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910b3a-7e7e-43d1-aff8-1fe61f46537c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.499933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96910b3a-7e7e-43d1-aff8-1fe61f46537c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.523583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-96910b3a-7e7e-43d1-aff8-1fe61f46537c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:34.524969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-96910b3a-7e7e-43d1-aff8-1fe61f46537c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:34.527045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:34.542608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:34.543231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.543508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.543846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.544702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] RBAC: Adding query filter params (role.id=bc847504306847498ec5148b7e5db940, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=a130eade9ce34454bf97655e0f9d4590) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:34.544875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:34.546307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:34.552460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 63/116] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:34 2022] GET /identity/v3/role_assignments?user.id=a130eade9ce34454bf97655e0f9d4590&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=bc847504306847498ec5148b7e5db940 => generated 259 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:34.564956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:34.642923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 54/117] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:34.645089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c9206-5a20-4402-8b1e-b981a52bc652 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.645878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c9206-5a20-4402-8b1e-b981a52bc652 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.646079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037c9206-5a20-4402-8b1e-b981a52bc652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.671850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-037c9206-5a20-4402-8b1e-b981a52bc652 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:34.675777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-037c9206-5a20-4402-8b1e-b981a52bc652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:34.792780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6d995-89f0-48cb-ae24-6705577287a4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:34.800258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 55/118] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:34.802628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167edbe-22ed-4d3f-9ce9-dc1db7681044 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.802628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167edbe-22ed-4d3f-9ce9-dc1db7681044 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.802628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167edbe-22ed-4d3f-9ce9-dc1db7681044 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.804007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 56/119] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:34.813935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.813935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.813935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.864639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:34.866026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:34.897955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:34.912039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:34.913034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:34.919961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:34.920175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:34.921733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:34.923260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:34.925924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 64/120] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:34 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:34.936816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:35.051793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 57/121] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:35.069103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:35.086332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:35.223674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcaae9-0781-453a-b027-abb03f06f2d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:35.303556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35781ecc-412b-4559-8809-6fd4262251b0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:35.319145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:35.319145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:35.319441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:35.319726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/7f8be41948d446bead6b1ad550c58d55/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:35.334130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7b19705b76444eba9e2a0476a6ff3e, user_id=7f8be41948d446bead6b1ad550c58d55, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:35.335630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:35.351771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:35.354535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:35.354795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:35.355875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:35.360882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 65/122] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:34 2022] PUT /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/7f8be41948d446bead6b1ad550c58d55/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:35.362386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:35.407777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:35.416876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:35.418248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:35.419039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 58/123] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:30:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:35.429515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:35.445307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:35.703482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7ccee-2770-4e5b-8354-701051a246cc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:35.763412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:35.764610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:35.765537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:35.766276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:35.767213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:35.767733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:35.770207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:35.789635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d947f36-8757-4612-9c62-11e75860c900 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:35.794459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 59/124] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:30:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:35.803583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:35.827729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:35.863424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:35.863424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:35.863424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:35.863424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:35.877248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:35.886077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:35.896529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 66/125] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 527 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:35.897332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:35.898216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:35.952480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6f7e-bb7b-4e33-a906-a7fe80a6bd61 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:35.997890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b9f92-1914-40db-aa56-a713185e8f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:35.998808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b9f92-1914-40db-aa56-a713185e8f4d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:35.999252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b9f92-1914-40db-aa56-a713185e8f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:35.999677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b9f92-1914-40db-aa56-a713185e8f4d None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.002182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b9f92-1914-40db-aa56-a713185e8f4d None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:36.004025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b9f92-1914-40db-aa56-a713185e8f4d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:36.005956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 60/126] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:35 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:36.007414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e959f-ea03-41ac-9481-b50a3575572c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.013694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e959f-ea03-41ac-9481-b50a3575572c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.014287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e959f-ea03-41ac-9481-b50a3575572c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.015228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033f5f8-21ba-434f-924c-d72fa83330cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:36.016428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 61/127] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:36.017497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e959f-ea03-41ac-9481-b50a3575572c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:36.058474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:36.059138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.059464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.059730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.074201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:36.080158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:36.081693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:36.083768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:36.086737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 67/128] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:36.087865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:36.148935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:36.228285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e959f-ea03-41ac-9481-b50a3575572c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:36.241286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0991d-7124-4db6-b5a5-dabd63b2c821 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:36.303880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:36.305769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.306136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.306509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.307287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:36.307693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:36.309221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beaa599-b789-440f-a8d5-57c36aff1cb3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:36.314167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:36.323508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 62/129] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:36.325134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.332906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.332906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.349944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:36.350995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.351913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.352598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.356652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:36.359343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:36.362788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 68/130] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:36 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:36.364012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:36.366138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:36.366305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:36.531781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 63/131] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:36.531781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:36.693933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b0036-de6e-4d70-a684-f2479fb5991a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:36.698451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:36.807366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:36.809512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.809797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.812532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.812532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:36.812532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:36.817753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:36.833957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 69/132] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:36.835194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:36.850734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef4371-2886-4b55-8d56-84edd34e518e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:36.902229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:36.924824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:36.926463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:36.928541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:36.929215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:36.931791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:36.933700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:36.936120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 64/133] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:36 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:36.937670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:37.183153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9fedb-2984-48c6-91b9-eec2f8b5eb25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:37.249719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdfe036-5b18-49b6-97f1-b407fdac31a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:37.274295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:37.275400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:37.275732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:37.276174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/7f8be41948d446bead6b1ad550c58d55/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:37.286071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7b19705b76444eba9e2a0476a6ff3e, user_id=7f8be41948d446bead6b1ad550c58d55, role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:37.287937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:37.322159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 70/134] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:36 2022] PUT /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/7f8be41948d446bead6b1ad550c58d55/roles/7682e93d1a8e439ba530f0911772aa20 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:37.323240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:37.366362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:37.367387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:37.367718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:37.368017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:37.372539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:37.374329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:37.384138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 65/135] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:36 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 447 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:37.387676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:37.420670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:37.536307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:37.578074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1dd296-f501-42aa-89fb-4bec02e403cb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:37.621349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:37.622884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:37.623574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:37.624160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:37.625869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:37.631782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:37.632745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 66/136] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:37.633581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:37.662264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939f0c0-d0bf-4825-9679-d7d11490a12e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:37.739818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64016217-8d44-4f2c-a38d-437f4823b6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:37.739818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64016217-8d44-4f2c-a38d-437f4823b6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:37.739818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64016217-8d44-4f2c-a38d-437f4823b6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:37.739818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64016217-8d44-4f2c-a38d-437f4823b6e7 None admin] PATH_INFO: `/v3/projects/9362c10e8edb48d1878f5e153f269516` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:37.740248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64016217-8d44-4f2c-a38d-437f4823b6e7 None admin] RBAC: Authorizing `identity:get_project(project_id=9362c10e8edb48d1878f5e153f269516)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:37.740248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64016217-8d44-4f2c-a38d-437f4823b6e7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:37.742449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 71/137] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:37 2022] GET /identity/v3/projects/9362c10e8edb48d1878f5e153f269516 => generated 312 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:37.749750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c88c3-ada1-40ca-aba6-36f22acd8b51 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:37.751978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c88c3-ada1-40ca-aba6-36f22acd8b51 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:37.751978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c88c3-ada1-40ca-aba6-36f22acd8b51 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:37.751978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 72/138] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:37.753155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c88c3-ada1-40ca-aba6-36f22acd8b51 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:37.786263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d5142-df86-47c9-822a-bdcac40a1b5c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:37.819880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:37.829281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:37.829944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:37.830263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:37.835770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:37.835770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:37.835770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:37.835770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 67/139] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:37 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:37.841575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:37.978900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a7715-56f6-4a4b-8ea9-db7dc199069e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:37.980880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:37.998676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c88c3-ada1-40ca-aba6-36f22acd8b51 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:38.072824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02e306-c640-4385-8c1e-c8b18a92d106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:38.075482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02e306-c640-4385-8c1e-c8b18a92d106 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.075695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02e306-c640-4385-8c1e-c8b18a92d106 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.076017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02e306-c640-4385-8c1e-c8b18a92d106 None admin] PATH_INFO: `/v3/projects/9362c10e8edb48d1878f5e153f269516/users/7f8be41948d446bead6b1ad550c58d55/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:38.081259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02e306-c640-4385-8c1e-c8b18a92d106 None admin] RBAC: Authorizing `identity:create_grant(project_id=9362c10e8edb48d1878f5e153f269516, user_id=7f8be41948d446bead6b1ad550c58d55, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:38.083902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02e306-c640-4385-8c1e-c8b18a92d106 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:38.100961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:38.101557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.103230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.103230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:38.105761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:38.108520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:38.109703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 68/140] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:37 2022] PUT /identity/v3/projects/9362c10e8edb48d1878f5e153f269516/users/7f8be41948d446bead6b1ad550c58d55/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:38.111330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a2a96-e11a-43d2-9fb1-26021ffca7cf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.112527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a2a96-e11a-43d2-9fb1-26021ffca7cf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.113043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a2a96-e11a-43d2-9fb1-26021ffca7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:38.156806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f73a2a96-e11a-43d2-9fb1-26021ffca7cf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:38.159448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f73a2a96-e11a-43d2-9fb1-26021ffca7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:38.167893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 73/141] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:30:37 2022] POST /identity/v3/users => generated 283 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:38.168933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:38.257373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:38.284856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 69/142] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:38.288061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc009c1f-0960-469c-bfd9-82a7d2580149 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.288767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc009c1f-0960-469c-bfd9-82a7d2580149 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.289510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc009c1f-0960-469c-bfd9-82a7d2580149 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:38.292157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 70/143] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:38.319013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.319992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.320652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:38.369500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:38.370716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:38.422355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846cbcfe-2115-4a81-ae3d-0b78600a1693 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:38.518211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:38.518820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.519106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.519560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/66bbceeaa7394d2b922231b2c11b4f8f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:38.526402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7b19705b76444eba9e2a0476a6ff3e, user_id=66bbceeaa7394d2b922231b2c11b4f8f, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:38.529068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:38.548691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 71/144] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:38.573572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 74/145] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:38 2022] PUT /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/66bbceeaa7394d2b922231b2c11b4f8f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:38.714384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:38.901920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08534e2-8f9f-49d5-ba98-2ffd8dbd67b3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:38.907174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:38.927903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:38.947141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:38.979480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:38.984181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:38.984809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:38.984809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.001864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:39.007067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:39.008055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:39.009616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 72/146] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:39.014572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:39.157933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7565e-3feb-4bb1-9254-be80700c83c2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:39.170203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a35430-a8ec-4e5a-846c-22328b763ade None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:39.203065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:39.203698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:39.203994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:39.204269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.204953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:39.205845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:39.207448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:39.217845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 73/147] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:39.225250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:39.249233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:39.249910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:39.255857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:39.255857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.278506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:39.286879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:39.292475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:39.293401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:39.294575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 75/148] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:39.303772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:39.355721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662d696-34a5-4930-8920-6abbc0b83873 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:39.398826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:39.399529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:39.399845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:39.400120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] PATH_INFO: `/v3/users/a130eade9ce34454bf97655e0f9d4590` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.401367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] RBAC: Authorizing `identity:get_user(user_id=a130eade9ce34454bf97655e0f9d4590)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:39.404469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:39.405053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 74/149] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:39 2022] GET /identity/v3/users/a130eade9ce34454bf97655e0f9d4590 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:39.415087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:39.497105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:39.547931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae005053-b7eb-431a-8f0c-e8ebec0dfe30 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:39.559967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:39.570115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673ee22-ef15-479d-8469-c1c384cbd119 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:39.595039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:39.596105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:39.596606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:39.597099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.598605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:39.600396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:39.603194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 75/150] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:39 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:39.609722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:39.688604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:39.689257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:39.689597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:39.690381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.691089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:39.691390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:39.692867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:39.717492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 76/151] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:39.724184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:39.846289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce1b20-05fe-4320-946e-55a45e058489 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:39.926377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:39.927071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:39.927349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:39.927625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/a130eade9ce34454bf97655e0f9d4590/roles/bc847504306847498ec5148b7e5db940` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:39.931994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7b19705b76444eba9e2a0476a6ff3e, user_id=a130eade9ce34454bf97655e0f9d4590, role_id=bc847504306847498ec5148b7e5db940)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:39.933468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:39.971477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 76/152] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:39 2022] PUT /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/users/a130eade9ce34454bf97655e0f9d4590/roles/bc847504306847498ec5148b7e5db940 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:40.063640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ee886-a3d6-4fec-affe-708630521ec7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:40.152216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:40.160872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:40.161648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:40.161928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:40.162199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:40.164288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:40.166850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:40.170979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 77/153] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:39 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:40.181437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:40.288336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:40.472181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da722354-ad92-4e0b-b1bf-0bebefaab883 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:40.542069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:40.542620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:40.542900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:40.543171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:40.697141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:40.698614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:40.777008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:40.922164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 77/154] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 741 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:40.928035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:41.153448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5c815-34f1-46d8-9598-ee6b7feb7bbe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:41.223031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aaddf-9c30-4d46-9bf3-92c2b84ec30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:41.223801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aaddf-9c30-4d46-9bf3-92c2b84ec30e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:41.224466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aaddf-9c30-4d46-9bf3-92c2b84ec30e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:41.224466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aaddf-9c30-4d46-9bf3-92c2b84ec30e None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:41.228501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7aaddf-9c30-4d46-9bf3-92c2b84ec30e None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:41.229999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7aaddf-9c30-4d46-9bf3-92c2b84ec30e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:41.231903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 78/155] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:40 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:41.238229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:41.425367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b63dd-00d3-4d34-9efa-a665e8399054 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:41.485599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:41.486185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:41.486471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:41.486737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:41.487910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=7f8be41948d446bead6b1ad550c58d55) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:41.487910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:41.489415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179a4c-22d5-47ad-a584-78e1d51ed544 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:41.492899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:41.499358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 78/156] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:41 2022] GET /identity/v3/role_assignments?user.id=7f8be41948d446bead6b1ad550c58d55&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:41.547982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ff139-fb7b-4200-a76b-907ade3d47f5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:41.548290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ff139-fb7b-4200-a76b-907ade3d47f5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:41.548538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ff139-fb7b-4200-a76b-907ade3d47f5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:41.549968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 79/157] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:41.556696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:41.556981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:41.557246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:41.583419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:41.584681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:41.699790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 79/158] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:42.420322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ff139-fb7b-4200-a76b-907ade3d47f5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:42.541809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:42.604817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ff139-fb7b-4200-a76b-907ade3d47f5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:42.661619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:42.661619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:42.661619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:42.661880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:42.681776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:42.686846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:42.687823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:42.688596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:42.692531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 80/159] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:30:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:42.698197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:42.867279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe4458-3fef-443b-814c-0ef7b7a2029f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:42.918955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:42.926619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:42.927178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:42.927453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:42.927725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:42.928443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:42.928744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:42.930178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:42.944357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 80/160] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:30:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:42.951781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:43.098433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9225a6c-9ef6-46f4-b426-f653b98292f3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:43.132175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:43.152675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdcee-1ba0-4751-911b-0ec8caaf2ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:43.153352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdcee-1ba0-4751-911b-0ec8caaf2ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.153631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdcee-1ba0-4751-911b-0ec8caaf2ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.153914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdcee-1ba0-4751-911b-0ec8caaf2ee4 None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.155247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdcee-1ba0-4751-911b-0ec8caaf2ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:43.156778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdcee-1ba0-4751-911b-0ec8caaf2ee4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:43.158930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 81/161] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:42 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:43.166624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:43.395119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99d2488-0385-4e0a-93ae-91f1093dfbe8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:43.450326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41675a-5fa0-4386-acb4-288d441cb883 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.450872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41675a-5fa0-4386-acb4-288d441cb883 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.450872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41675a-5fa0-4386-acb4-288d441cb883 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.452290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 82/162] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:43.465277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef720d3a-11da-4d61-84fc-173f98dd1d62 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.465565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef720d3a-11da-4d61-84fc-173f98dd1d62 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.465838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef720d3a-11da-4d61-84fc-173f98dd1d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.480492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:43.481128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.481403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.481660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.505564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ef720d3a-11da-4d61-84fc-173f98dd1d62 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:43.506809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef720d3a-11da-4d61-84fc-173f98dd1d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:43.630255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 83/163] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:43.631527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db27b8-7b96-4f1f-9b06-4e9e36bbeac1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.631995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db27b8-7b96-4f1f-9b06-4e9e36bbeac1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.632273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db27b8-7b96-4f1f-9b06-4e9e36bbeac1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.633659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 84/164] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:43.641252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6bf78-dbf4-4d75-aae4-9101c9c4a987 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.641534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6bf78-dbf4-4d75-aae4-9101c9c4a987 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.641807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6bf78-dbf4-4d75-aae4-9101c9c4a987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.677399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-70a6bf78-dbf4-4d75-aae4-9101c9c4a987 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:43.678620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-70a6bf78-dbf4-4d75-aae4-9101c9c4a987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:43.764645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:43.808906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:43.810377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:43.860609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 85/165] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:43.862794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629bea-0ce5-4aa2-af15-88a233ec1bf7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.863289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629bea-0ce5-4aa2-af15-88a233ec1bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.869864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629bea-0ce5-4aa2-af15-88a233ec1bf7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.871615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 86/166] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:43 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:43.877793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:43.878313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:43.878808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:43.924575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:43.926239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:44.014347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 81/167] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 848 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:44.016088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:44.069277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:44.103441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 87/168] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:44.105365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:44.332189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:44.375258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a51c3-505f-4057-9c1d-8a6a44c1dd2f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:44.395808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67361efd-6137-4c00-bc1d-4a8f83b6a1cb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:44.514923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa49453-7f14-4ffa-9b0a-38ac4112d917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:44.515571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49453-7f14-4ffa-9b0a-38ac4112d917 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:44.515991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49453-7f14-4ffa-9b0a-38ac4112d917 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:44.517275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49453-7f14-4ffa-9b0a-38ac4112d917 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:44.517490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:44.519777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:44.519777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:44.519777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:44.526498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:44.527582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:44.528721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 88/169] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:44 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:44.535201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:44.549368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa49453-7f14-4ffa-9b0a-38ac4112d917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3aa49453-7f14-4ffa-9b0a-38ac4112d917 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:44.555657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:44.556617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:44.557410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:44.559682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 82/170] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 545 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:44.562475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d0e05-45aa-4e0c-bc4f-954c3d4d4386 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:44.562759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d0e05-45aa-4e0c-bc4f-954c3d4d4386 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:44.563557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856d0e05-45aa-4e0c-bc4f-954c3d4d4386 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:44.566302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 83/171] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:44.567136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d0e05-45aa-4e0c-bc4f-954c3d4d4386 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:44.858963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:44.897943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856d0e05-45aa-4e0c-bc4f-954c3d4d4386 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:44.911981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f0f0b-77d5-466e-bdad-e85a59e31226 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:44.938616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:44.940085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:44.940366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:44.940641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:44.941376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] RBAC: Adding query filter params (role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=7f8be41948d446bead6b1ad550c58d55) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:44.941846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:44.943301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:44.950137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 84/172] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:44 2022] GET /identity/v3/role_assignments?user.id=7f8be41948d446bead6b1ad550c58d55&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 608 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:44.950327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411d02-4855-4d35-a6af-a05fa77d30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:44.951028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411d02-4855-4d35-a6af-a05fa77d30a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:44.951287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:44.951387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411d02-4855-4d35-a6af-a05fa77d30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:44.951549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411d02-4855-4d35-a6af-a05fa77d30a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:44.975777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e411d02-4855-4d35-a6af-a05fa77d30a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3e411d02-4855-4d35-a6af-a05fa77d30a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:44.981682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:44.982658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:44.983477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 89/173] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:44.984716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:44.984900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:44.985134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:44.990409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.005064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:45.006299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:45.083260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 90/174] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:45.089215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.096048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092427c-b00a-4c9a-a193-7aaea2ed4f9a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.139868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.140723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.141004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.141278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.141962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:45.142304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:45.143703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c08f56-2abd-4bf6-a2f1-3ef977402947 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:45.159966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 85/175] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:45.161770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a3c7b-148e-4f7b-afe6-8752799bc51b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.162057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a3c7b-148e-4f7b-afe6-8752799bc51b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.162324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a3c7b-148e-4f7b-afe6-8752799bc51b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.163659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 86/176] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:45.165372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f8d1f-284e-414b-baa0-e5a32f976e2a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.165664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f8d1f-284e-414b-baa0-e5a32f976e2a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.165931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6f8d1f-284e-414b-baa0-e5a32f976e2a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.167807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 87/177] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:45.168887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f8d1f-284e-414b-baa0-e5a32f976e2a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.170965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.247608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f9b2-fff2-424f-8cbb-ce6e75c26d3b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.289164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.296840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.297474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.297749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.298016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.309775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:45.314621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:45.315700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 91/178] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:45.316530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.317920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6f8d1f-284e-414b-baa0-e5a32f976e2a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.364421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.365037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.365315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.365610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.366280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:45.366580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:45.367972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af9f29-4722-4208-bc41-3a04b93de272 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:45.376647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 88/179] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:45.377682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a20f45-f8ff-4645-8368-3eb23d1da74a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.377965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a20f45-f8ff-4645-8368-3eb23d1da74a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.378236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a20f45-f8ff-4645-8368-3eb23d1da74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.397573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d6a20f45-f8ff-4645-8368-3eb23d1da74a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:45.398676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6a20f45-f8ff-4645-8368-3eb23d1da74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:45.430982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c66d7a-7094-45e8-8460-13275a5dd1e5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.433285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.466264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.466892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.467161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.467427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] PATH_INFO: `/v3/users/66bbceeaa7394d2b922231b2c11b4f8f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.468552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] RBAC: Authorizing `identity:get_user(user_id=66bbceeaa7394d2b922231b2c11b4f8f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:45.468654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 89/180] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:45.469882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.470055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:45.470165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.470978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.472330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 92/181] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/users/66bbceeaa7394d2b922231b2c11b4f8f => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:45.472908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.497393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:45.498664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:45.575091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.583063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 90/182] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:45.584623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.616240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb681b8-55fd-42ac-bc37-9efc835bcf81 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.656664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.657588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.658753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.660546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.661653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:45.662050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:45.663824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:45.675903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 93/183] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:45.677680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.726193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.730683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698fbea-97d5-48c5-b419-345dfa7486db None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.792689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.793404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.793682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.793965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] PATH_INFO: `/v3/users/7f8be41948d446bead6b1ad550c58d55` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.795367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8be41948d446bead6b1ad550c58d55)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:45.797037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:45.798701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 91/184] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/users/7f8be41948d446bead6b1ad550c58d55 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:45.800054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.816126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800870b2-c37f-4cb6-b5c1-1463f4b4052d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.861815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.862952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.863423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.863904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:45.877106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:45.883040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:45.883976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:45.884756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:45.887303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 94/185] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:45.895229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:45.913111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:45.951591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc7c3-faeb-4ea2-b2cc-b7c32bf9edf7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:45.991978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:45.992892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:45.993165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:45.993513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.001930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.009712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f85d9-9150-4686-b322-bad5235a3c9b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.047927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.049659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.050238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.050657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] PATH_INFO: `/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.065647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] RBAC: Authorizing `identity:get_user(user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.067136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.068843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 95/186] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:46.069617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.075707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.077225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.146810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.184306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 92/187] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:46.188123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2ff17-962e-4347-a9b4-09b7aa6a3a33 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.189778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.223109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.223734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.224029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.224298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.225377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.226821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.228549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 96/188] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:46.229124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.253341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.292832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7f680-3070-4d70-b5c6-cfb82ba8423c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.328101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.328704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.328977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.329245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.330782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e02faf-10ab-46ba-bcc0-c9ee45e0636f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.363773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.365172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.365806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.366062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.366334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.367008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:46.367308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.368734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.377009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 97/189] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:46.377837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.398187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.399639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.457519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.477479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa4951-4592-41d6-8bfa-008fd4d0496c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.497400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 93/190] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:46.498265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.513638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.514268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.514542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.514812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.525322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:46.530541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:46.531596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:46.532721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 98/191] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:46.534046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.567508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.603104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742a572-4f96-4709-8b47-081c52fd1b90 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.633106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d818758-8b4d-42c8-a4b0-8959fef38b40 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.635552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.636191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.636466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.636736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.667420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.668239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.668569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.668851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.670068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.671554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.673386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 99/192] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:46.674184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.687873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.710780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.712371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.780296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321115d4-d31f-4cf8-b205-7025ae6e37f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.782439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.815461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:46.816089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:46.816356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:46.816621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:46.817276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:46.819058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:46.822182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 94/193] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:46.823030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.844316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 100/194] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:30:46 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:46.845287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:46.929092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:46.959837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239cffd6-e1b4-40d1-b3d7-1bc93cadb742 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:46.978679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee72ba-f23f-47b5-a166-416adbe208f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.000881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.001589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.001865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.002137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] PATH_INFO: `/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.003344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] RBAC: Authorizing `identity:get_user(user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.004873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.006548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 95/195] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.007430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.022770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.023425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.023704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.024031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] PATH_INFO: `/v3/projects/9362c10e8edb48d1878f5e153f269516` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.025323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] RBAC: Authorizing `identity:get_project(project_id=9362c10e8edb48d1878f5e153f269516)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.027022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.028559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 101/196] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:46 2022] GET /identity/v3/projects/9362c10e8edb48d1878f5e153f269516 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.029416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.069066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:47.165106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5561e-df5b-46ed-88e2-f8d3eeacc925 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.194313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c634c-d478-4aa2-8515-50646a81c35c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.211586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.212293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.212832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.213096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.213757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:47.214053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.215414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.227883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 96/197] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.228525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.247692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.248259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.248538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.248807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.249538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=66bbceeaa7394d2b922231b2c11b4f8f) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:47.249842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.251313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.253350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:47.260080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 102/198] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/role_assignments?user.id=66bbceeaa7394d2b922231b2c11b4f8f&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.260787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.355208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:47.359626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9cccf-f7e3-43de-b57b-e2afa97350fd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.399565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.400282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.400555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.400823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.401985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.403440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.405180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 97/199] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.405982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.419645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f399ff4-b902-4036-ba02-dd4cf34133f2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.485372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.486038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.486311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.486577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.558056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c53025-2c15-4560-9cba-753716ba3148 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.583558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:47.589551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.591789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.603719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.604365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.604646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.604912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.605647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=9362c10e8edb48d1878f5e153f269516, user.id=7f8be41948d446bead6b1ad550c58d55) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:47.605948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.607378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.615624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 98/200] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/role_assignments?user.id=7f8be41948d446bead6b1ad550c58d55&scope.project.id=9362c10e8edb48d1878f5e153f269516&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.616495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.696613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:47.722614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 103/201] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:47.723512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.868280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:47.876340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523474b7-f385-493d-924b-a109a1df6d1a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.893215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7887fd-e36a-4b5d-9d08-817f64767619 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:47.936970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.937724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.937998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.938255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.940001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:47.940309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.942539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.950525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 104/202] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/role_assignments?user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.951353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:47.953318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:47.954305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:47.954618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:47.955943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] PATH_INFO: `/v3/users/a130eade9ce34454bf97655e0f9d4590` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:47.957287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] RBAC: Authorizing `identity:get_user(user_id=a130eade9ce34454bf97655e0f9d4590)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:47.958795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:47.960626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 99/203] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/users/a130eade9ce34454bf97655e0f9d4590 => generated 289 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:47.967640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:48.149399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:48.156138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4c01b-82f0-47b7-ba13-d92f80e7a951 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:48.179070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49f350-434f-4d9e-95cf-ae5dcc8a324f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:48.226266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:48.226891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:48.227231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:48.227549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:48.238430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:48.239393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:48.239923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:48.242723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:48.244290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:48.245976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:48.248166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 105/204] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:48.268794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:48.306398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:48.308005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:48.354621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:48.408609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:48.412969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 100/205] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:30:48.418851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:48.530548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e80f5f-6c66-4599-95e7-b66460eb9536 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:48.533688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01569b-86bf-4379-941f-7e0ecedeede7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:48.570457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:48.571118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:48.571389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:48.571657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:48.575771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:48.575771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:48.576256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 101/206] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:48 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:48.583022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:48.628485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:48.629455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:48.629932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:48.630421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:48.631352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] RBAC: Adding query filter params (role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:48.631916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:48.633581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a42fc-bb20-456a-9e26-6750e28f026c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:48.649627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 106/207] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:48 2022] GET /identity/v3/role_assignments?user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 259 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:48.661816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:48.711097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99c851-8a56-41d4-bdcd-602c86dfc1ba None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:48.752159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:48.752928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:48.753430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:48.753727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:48.754618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] RBAC: Adding query filter params (role.id=bc847504306847498ec5148b7e5db940, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e, user.id=a130eade9ce34454bf97655e0f9d4590) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:48.754940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:48.756552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade0732-cae7-4342-b393-474c8c94f5a6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:48.764147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 102/208] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:48 2022] GET /identity/v3/role_assignments?user.id=a130eade9ce34454bf97655e0f9d4590&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=bc847504306847498ec5148b7e5db940 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:49.441908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c45ad-7e5b-4950-b900-20bfc7940629 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:49.442943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c45ad-7e5b-4950-b900-20bfc7940629 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:49.443444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c45ad-7e5b-4950-b900-20bfc7940629 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:49.445049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 107/209] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:49.450926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149927-fb00-485c-a14e-237c290001ae None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:49.451387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149927-fb00-485c-a14e-237c290001ae None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:49.451485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba149927-fb00-485c-a14e-237c290001ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:49.472167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ba149927-fb00-485c-a14e-237c290001ae None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:49.473360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ba149927-fb00-485c-a14e-237c290001ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:49.548848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 103/210] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:49.729598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b368-bf8c-49bb-82c7-c948f54f9db3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:49.729890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b368-bf8c-49bb-82c7-c948f54f9db3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:49.730088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457b368-bf8c-49bb-82c7-c948f54f9db3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:49.731675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 108/211] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:49.741786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:49.742213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:49.742586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:49.766116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:49.767920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:49.827067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b368-bf8c-49bb-82c7-c948f54f9db3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:49.842000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:49.856543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 104/212] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:49.950848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:49.960437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457b368-bf8c-49bb-82c7-c948f54f9db3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.000454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.001055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.001830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.002091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.013257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:50.018804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:50.021098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:50.022973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 109/213] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:50.028992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.147374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b101d2-2e65-4a3a-8644-c09ed36d2bf2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.184210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.184824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.185092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.185358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.186022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:50.186315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:50.187685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:50.197511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 105/214] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:50.208859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.227386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.291321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:50.339907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc95e5-d120-4c47-a2d9-7fd7ddc67eb9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.346258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac700cdb-9c27-495b-b78d-2d5b6b8c4b91 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.383717 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.384961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.385251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.385526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] PATH_INFO: `/v3/users/146d20c1436542f8908f05efc2452ff1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.390560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.391610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.392640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.393111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.394513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:50.396291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:50.398755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 106/215] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:50.404168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.405894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] RBAC: Authorizing `identity:get_user(user_id=146d20c1436542f8908f05efc2452ff1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:50.408314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:50.410147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 110/216] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:50 2022] GET /identity/v3/users/146d20c1436542f8908f05efc2452ff1 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:50.415634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.435688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:50.523024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea37b00-afff-4b34-ad82-5e3bb20df710 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.543718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:50.543914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b200f-8794-4c55-b0fb-e9d5bf70e38a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.559357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a99f0-6e48-4747-bba1-ba193c461355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.559994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a99f0-6e48-4747-bba1-ba193c461355 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.560268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a99f0-6e48-4747-bba1-ba193c461355 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.560535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a99f0-6e48-4747-bba1-ba193c461355 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.561184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a99f0-6e48-4747-bba1-ba193c461355 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:50.562625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a99f0-6e48-4747-bba1-ba193c461355 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:50.585472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.586093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.586360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.586625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.591061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 107/217] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:30:50 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:50.592386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44abf5-e2e1-40d5-8fe5-70f19ac2c5b8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.592664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44abf5-e2e1-40d5-8fe5-70f19ac2c5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.592932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44abf5-e2e1-40d5-8fe5-70f19ac2c5b8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.594330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 108/218] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:50.600029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.600352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.600570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.620993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:50.622199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:50.706609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:50.708173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 109/219] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:50.708856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:50.726769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:50.826391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 111/220] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:30:50.833290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.944704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0870d7-fcd1-4e7a-9d85-90b82b57cac0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:50.953718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:50.987805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:50.988482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:50.988816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:50.989102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:50.990544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:50.992496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:50.997958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:50.997958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 110/221] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:50 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:51.005523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.125147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bffb24-7c13-4885-8c84-f979f2cd8452 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:51.141695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:51.142318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d946e7-8830-474d-a880-1bc981946b2a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:51.175198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc105f83-93e1-4485-b6ae-3a7e7fc3d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:51.176016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc105f83-93e1-4485-b6ae-3a7e7fc3d714 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.176303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc105f83-93e1-4485-b6ae-3a7e7fc3d714 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.176872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc105f83-93e1-4485-b6ae-3a7e7fc3d714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.189939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:51.190561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.190834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.191096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.192024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=146d20c1436542f8908f05efc2452ff1) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:51.192463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:51.192670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc105f83-93e1-4485-b6ae-3a7e7fc3d714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:51.193975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dc105f83-93e1-4485-b6ae-3a7e7fc3d714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:51.199013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:51.199939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:51.201335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 112/222] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:51.201989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2709f-fe1b-4c67-b83a-b86e90ec801c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.202063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2709f-fe1b-4c67-b83a-b86e90ec801c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.202335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2709f-fe1b-4c67-b83a-b86e90ec801c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.204477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 113/223] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:51.208111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 111/224] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:51 2022] GET /identity/v3/role_assignments?user.id=146d20c1436542f8908f05efc2452ff1&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:51.210020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.210223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.210412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.210564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.233467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:51.234737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:51.304162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:51.400424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 114/225] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:51.404128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0a387-07b1-4e7e-ac4f-3328c03d58fe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:51.448207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:51.448873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.449151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.449423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.450220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:51.450530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:51.451985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:51.463420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 112/226] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:51.469256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.600545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fecad-c131-4ccc-a6bf-869eaf7cf1d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:51.602308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:51.650337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:51.651001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.651224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.651488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] PATH_INFO: `/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.652711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] RBAC: Authorizing `identity:get_user(user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:51.654280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:51.656594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 115/227] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:51 2022] GET /identity/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:51.662392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.702171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.865188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741a8d5-a109-4b12-b9d2-be49d4ff2f6f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:51.869916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:51.878310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866e244-8d7c-45b3-a9b0-16d3a77d41fa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:51.928596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:51.929245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.929521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.929936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.941876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:51.947503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:51.948460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:51.949694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 116/228] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:30:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:30:51.955051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.961372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:51.962580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:51.963071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:51.963560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:51.966283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:51.967998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:51.970486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 113/229] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:51 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:51.976152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:51.991102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:52.107774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa69c32-a1eb-4117-bed8-93d03167cf1a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:52.147638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:52.148297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:52.148654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:52.155778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:52.155778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:52.155778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:52.155778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:52.163429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 117/230] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:30:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:52.174354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:52.280218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:52.285747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00e615-9f46-4dc9-a964-c80789cffc92 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:52.314044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6878c8f-014a-49ac-8414-dfd403635f22 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:52.332063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:52.333069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:52.333549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:52.334073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] PATH_INFO: `/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:52.335486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] RBAC: Authorizing `identity:get_user(user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:52.337472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:52.339618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 118/231] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:52 2022] GET /identity/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:52.346228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:52.386684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:52.387400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:52.387683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:52.387988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/9f7c2e77f99f4d8c9fc9e003d3331eb7/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:52.393418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] RBAC: Authorizing `identity:create_grant(project_id=e22dee573a0047c6958f6c34395e3b67, user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:52.394890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:52.422250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 114/232] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:51 2022] PUT /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/9f7c2e77f99f4d8c9fc9e003d3331eb7/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:30:52.464158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba238bb-aba0-4875-afe4-4ef405c3f62f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:52.504246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23221a-8cf0-4acc-b959-41c760fdb014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:52.505270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23221a-8cf0-4acc-b959-41c760fdb014 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:52.505818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23221a-8cf0-4acc-b959-41c760fdb014 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:52.506317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23221a-8cf0-4acc-b959-41c760fdb014 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:52.507702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23221a-8cf0-4acc-b959-41c760fdb014 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:52.509411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23221a-8cf0-4acc-b959-41c760fdb014 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:52.511590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 119/233] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:52 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:52.518814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:52.556423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:52.646957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea3fc3-3b5e-4802-9cf0-1217139b2cf3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:52.718406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c78b4-1fa6-41e9-be02-397060c265df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:52.720959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c78b4-1fa6-41e9-be02-397060c265df None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:52.721793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c78b4-1fa6-41e9-be02-397060c265df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:52.722575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c78b4-1fa6-41e9-be02-397060c265df None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/9f7c2e77f99f4d8c9fc9e003d3331eb7/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:52.731776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c78b4-1fa6-41e9-be02-397060c265df None admin] RBAC: Authorizing `identity:create_grant(project_id=e22dee573a0047c6958f6c34395e3b67, user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7, role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:52.735297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c78b4-1fa6-41e9-be02-397060c265df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:52.760781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 115/234] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:52 2022] PUT /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/9f7c2e77f99f4d8c9fc9e003d3331eb7/roles/7682e93d1a8e439ba530f0911772aa20 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:53.444161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b0f38-9385-4445-af8c-b7af6b830c3a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:53.444685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b0f38-9385-4445-af8c-b7af6b830c3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:53.444685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b0f38-9385-4445-af8c-b7af6b830c3a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:53.446394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 120/235] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:53.451885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb016a-0b8b-4714-86c5-6d1121796dca None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:53.452175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb016a-0b8b-4714-86c5-6d1121796dca None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:53.452435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb016a-0b8b-4714-86c5-6d1121796dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:53.472732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d2cb016a-0b8b-4714-86c5-6d1121796dca None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:53.473933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d2cb016a-0b8b-4714-86c5-6d1121796dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:53.549150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 116/236] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:53.893374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b0f38-9385-4445-af8c-b7af6b830c3a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:53.905199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444f5d-6ae7-4c89-9d17-6a634c112a19 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:53.905717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444f5d-6ae7-4c89-9d17-6a634c112a19 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:53.905717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2444f5d-6ae7-4c89-9d17-6a634c112a19 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:53.907114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 117/237] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:53.914451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99f1de-e973-46da-ab2b-6bbfd08cd43b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:53.914762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99f1de-e973-46da-ab2b-6bbfd08cd43b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:53.915003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df99f1de-e973-46da-ab2b-6bbfd08cd43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:53.952783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:53.954842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-df99f1de-e973-46da-ab2b-6bbfd08cd43b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:53.956096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-df99f1de-e973-46da-ab2b-6bbfd08cd43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:53.981191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:54.089527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 118/238] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:54.127488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ff180-38d8-4e50-9aa5-1689274d18e0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.127833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ff180-38d8-4e50-9aa5-1689274d18e0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.128081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ff180-38d8-4e50-9aa5-1689274d18e0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:54.130796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 119/239] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:54.134175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b0f38-9385-4445-af8c-b7af6b830c3a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:54.140085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.145496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.146070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:54.177993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:54.179188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:54.208751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:54.209776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.210256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.210717 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:54.249614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:54.262697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:54.263670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:30:54.264491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:54.267395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 121/240] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:54.276363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:54.280825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 120/241] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:54.404352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:54.419775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721ce35-fade-4a3b-a850-2564890b5c4b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:54.456322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:54.457606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.458051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.458460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:54.459271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:54.459716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:54.461257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:54.471539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 122/242] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:54.477628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:54.576200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:54.618500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce516-ab4c-478a-9595-50758f4909d1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:54.699712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:54.700801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.701272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.701739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] PATH_INFO: `/v3/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:54.738450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:54.738643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] RBAC: Authorizing `identity:get_user(user_id=09ee35e6dd2a4d89bb0fd9db86e1f2d7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:54.741277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:54.745179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 121/243] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:54 2022] GET /identity/v3/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7 => generated 297 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:54.757857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:54.884436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e856f0c-b9db-43ae-9851-b1aed1c33423 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:54.926589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:54.927637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.928136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.928601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:54.947889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad1557-d6f1-4c5f-b6dd-50fc1dd434f6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:54.948409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:54.960007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:54.960936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:30:54.961729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:54.962591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 123/244] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:54.963791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:54.990510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:54.996154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:54.997020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:54.997437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:54.997849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.002418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:55.078002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.080079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.178869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460565cc-6a1b-4b24-baa6-23c8a5223530 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:55.195078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 122/245] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:55.200996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:55.245004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:55.258968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:55.259610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.259930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.260170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.262019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.263499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.270304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 124/246] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:30:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:55.271248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:55.361503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:55.385400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7edf31-a449-4a23-b975-f295d26dae80 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:55.394530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a9d37-0aac-4ebc-b223-0ba3750b97f8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:55.426716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:55.427708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.428206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.428667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.430613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.433181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.441530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 125/247] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:55 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:55.441530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:55.445491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:55.446466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.446923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.447375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.448351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:55.448751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.451117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a997355-7711-42e6-9310-aed381633c3b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.464951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 123/248] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:30:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:55.467196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf522-6e9c-4f5d-aa77-688c2ae81b79 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.467653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf522-6e9c-4f5d-aa77-688c2ae81b79 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.468143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcf522-6e9c-4f5d-aa77-688c2ae81b79 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.469948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 124/249] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:55.471634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf522-6e9c-4f5d-aa77-688c2ae81b79 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:55.612267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab52ab2-c657-4ff1-9e07-7b7d9ace503d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:55.619028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:55.624810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcf522-6e9c-4f5d-aa77-688c2ae81b79 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:55.649198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:55.649810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.650085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.650354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.651010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.652497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.664390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:55.665266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.665541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.665815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.666545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] RBAC: Adding query filter params (role.id=bc847504306847498ec5148b7e5db940, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=09ee35e6dd2a4d89bb0fd9db86e1f2d7) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:55.666857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.668389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382a93f-6829-4f50-97d4-69f485922bd9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.674823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 126/250] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:30:55 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:30:55.675020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 125/251] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:55 2022] GET /identity/v3/role_assignments?user.id=09ee35e6dd2a4d89bb0fd9db86e1f2d7&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=bc847504306847498ec5148b7e5db940 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:55.676069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd6648-26e2-4d02-88dc-a67f3c011981 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.676354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd6648-26e2-4d02-88dc-a67f3c011981 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.676627 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd6648-26e2-4d02-88dc-a67f3c011981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.680823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:55.687566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:55.722818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b3fd6648-26e2-4d02-88dc-a67f3c011981 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:55.722818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b3fd6648-26e2-4d02-88dc-a67f3c011981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:55.810584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 126/252] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:55.811800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d995b-07e7-4313-89ef-86c79b236986 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.812084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d995b-07e7-4313-89ef-86c79b236986 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.812961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d995b-07e7-4313-89ef-86c79b236986 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.813742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 127/253] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:55.822300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.822549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.822678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.844920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:55.846161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:55.907630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8808a76-0b47-4ac9-8927-2ef549e09da3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:55.958449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:55.959101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:55.959369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:55.959629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] PATH_INFO: `/v3/users/146d20c1436542f8908f05efc2452ff1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:55.959958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 128/254] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:30:55.963777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] RBAC: Authorizing `identity:get_user(user_id=146d20c1436542f8908f05efc2452ff1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:55.963777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:55.964735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 127/255] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:55 2022] GET /identity/v3/users/146d20c1436542f8908f05efc2452ff1 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:55.971896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.032717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:56.154887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a8671-ca90-41e6-930e-4be7bdaee962 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:56.216065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:56.217086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:56.217550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:56.218007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:56.220561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:56.223250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:56.226283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 129/256] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:55 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:56.232523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.259840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.321886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:56.360810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ac1c9-9755-421b-8f9a-1369d839788a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:56.408428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:56.408714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:56.409010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:56.409149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/146d20c1436542f8908f05efc2452ff1/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:56.411108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e22dee573a0047c6958f6c34395e3b67, user_id=146d20c1436542f8908f05efc2452ff1, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:56.412613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:56.430409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 128/257] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:30:56 2022] PUT /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/146d20c1436542f8908f05efc2452ff1/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:30:56.431907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.451597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:56.474519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf25c49-158a-4661-8220-954ddc9654ab None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:56.539261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:56.540036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:56.540314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:56.540582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:56.558393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:56.567711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:56.569112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:30:56.570620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:30:56.572699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 130/258] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:30:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:56.577212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.597174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e92303-9759-4760-ac36-9b291d3495d2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:56.690689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:56.713600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed57c9-104b-46d8-9c73-09e4679a0c67 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:56.721335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:56.722015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:56.722294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:56.722566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:56.739439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:30:56.744866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:30:56.747218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:30:56.749155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 129/259] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:30:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:30:56.758124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.758766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:56.759409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:56.759682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:56.759996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:56.760676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:56.760982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:56.762393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:56.781583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:30:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:56.787278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:56.805919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:56.908045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327dbd-0a19-439c-bd8c-bb72198ba7c7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:56.946898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:56.947667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:56.947971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:56.948238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] PATH_INFO: `/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:56.949406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] RBAC: Authorizing `identity:get_user(user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:56.950881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:56.952857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 132/261] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:56 2022] GET /identity/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:56.958883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:57.060444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:57.079940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a1799-1caa-4df9-a2d6-0facdfe170b9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:57.119033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:57.119798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:57.120460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:57.120731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:57.159149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78801dc2-dc87-482e-ace9-8ec5cd254105 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:57.227817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:57.229596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:57.295577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019772-066a-40f7-a098-35903256ecad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:57.296238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019772-066a-40f7-a098-35903256ecad None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:57.296516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019772-066a-40f7-a098-35903256ecad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:57.296774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019772-066a-40f7-a098-35903256ecad None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:57.297514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019772-066a-40f7-a098-35903256ecad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:57.298331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019772-066a-40f7-a098-35903256ecad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:57.299736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019772-066a-40f7-a098-35903256ecad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:57.323736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 130/262] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 566 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:57.329761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019772-066a-40f7-a098-35903256ecad None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:57.384273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:57.426051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 133/263] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:30:57.431423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:57.445539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:57.544396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019772-066a-40f7-a098-35903256ecad None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:57.595417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:57.596451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:57.596869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:57.597280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] PATH_INFO: `/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:57.604817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] RBAC: Authorizing `identity:get_user(user_id=9f7c2e77f99f4d8c9fc9e003d3331eb7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:57.606572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:57.608449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 131/264] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:30:57 2022] GET /identity/v3/users/9f7c2e77f99f4d8c9fc9e003d3331eb7 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:57.616657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:57.778894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d8835-dadc-4735-935a-ecf9d7b0219e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:57.793046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bef67-131e-4f7d-b7bc-bac3998a84f2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:57.822917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:57.847325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:57.848480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:57.848900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:57.849366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:57.877288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:57.877951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:57.878229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:57.878532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:57.879668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:57.883801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:57.892693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 134/265] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:57 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:30:57.909684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:57.988348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:57.990128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:58.100363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:58.189448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 132/266] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:30:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:30:58.204395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:58.271486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bfe67e-3c29-47e4-b5a0-7552971b62d9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:58.273930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:58.388035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:58.389021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:58.389517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:58.390005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:58.391467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] RBAC: Adding query filter params (role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:58.392553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:58.395438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:58.421834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 135/267] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:57 2022] GET /identity/v3/role_assignments?user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 608 bytes in 513 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:58.529014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dbd29b-1958-445a-a18d-10332cfd0394 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:58.601427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adf41a-ed9e-461b-84fd-521c1cadf566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:58.602513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adf41a-ed9e-461b-84fd-521c1cadf566 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:58.603000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adf41a-ed9e-461b-84fd-521c1cadf566 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:58.603482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adf41a-ed9e-461b-84fd-521c1cadf566 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:58.605519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adf41a-ed9e-461b-84fd-521c1cadf566 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:58.608476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22adf41a-ed9e-461b-84fd-521c1cadf566 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:58.612844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 133/268] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:30:58 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:58.622227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:30:58.832447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d67d7-36c9-412e-9492-a7b1be5cf8da None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:30:58.852852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:30:58.886296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:30:58.886937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:58.887214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:58.887485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:58.888292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:30:58.888609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:30:58.890144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952eea6-cf88-4dc1-aa26-f205bf7e0b09 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:30:58.901649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 136/269] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:30:58 2022] GET /identity/v3/role_assignments?user.id=9f7c2e77f99f4d8c9fc9e003d3331eb7&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:30:59.491794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ffab5-141f-42ca-8aa9-9b3f62555514 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:59.491794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ffab5-141f-42ca-8aa9-9b3f62555514 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:59.491794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ffab5-141f-42ca-8aa9-9b3f62555514 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:59.493191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 134/270] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:30:59.499068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9f5e-57e4-4535-886f-210b4c29c5de None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:30:59.499558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9f5e-57e4-4535-886f-210b4c29c5de None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:30:59.499658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc9f5e-57e4-4535-886f-210b4c29c5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:30:59.522232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-84bc9f5e-57e4-4535-886f-210b4c29c5de None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:30:59.523433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-84bc9f5e-57e4-4535-886f-210b4c29c5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:30:59.607359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 137/271] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:30:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:30:59.906819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ffab5-141f-42ca-8aa9-9b3f62555514 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.015861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76249f61-0c81-4b23-9bd8-ed1679a45f09 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.016373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76249f61-0c81-4b23-9bd8-ed1679a45f09 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.016520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76249f61-0c81-4b23-9bd8-ed1679a45f09 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.018141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 138/272] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:00.023934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea5b6b-52c9-4cda-9e45-37bb68ef3218 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.024322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea5b6b-52c9-4cda-9e45-37bb68ef3218 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.024497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea5b6b-52c9-4cda-9e45-37bb68ef3218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.043798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8fea5b6b-52c9-4cda-9e45-37bb68ef3218 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:00.045037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8fea5b6b-52c9-4cda-9e45-37bb68ef3218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:00.107299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:00.139317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ffab5-141f-42ca-8aa9-9b3f62555514 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.160155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 139/273] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:00.160155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be8ea-d54a-4dbf-a104-f473dfdd1dfc None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.160155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be8ea-d54a-4dbf-a104-f473dfdd1dfc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.160155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17be8ea-d54a-4dbf-a104-f473dfdd1dfc None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.160155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 140/274] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:00.165606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b0ff1-d74f-4d2d-b7c8-e584a0857bfa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.165876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b0ff1-d74f-4d2d-b7c8-e584a0857bfa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.166149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b0ff1-d74f-4d2d-b7c8-e584a0857bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.190294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bb6b0ff1-d74f-4d2d-b7c8-e584a0857bfa None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:00.191628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bb6b0ff1-d74f-4d2d-b7c8-e584a0857bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:00.210868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed8e6c-abdb-4291-a2c6-4b7275e45a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.211544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed8e6c-abdb-4291-a2c6-4b7275e45a42 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.211871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed8e6c-abdb-4291-a2c6-4b7275e45a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.212400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed8e6c-abdb-4291-a2c6-4b7275e45a42 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.225561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed8e6c-abdb-4291-a2c6-4b7275e45a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-22ed8e6c-abdb-4291-a2c6-4b7275e45a42 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:00.230683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:00.238294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 135/275] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:30:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:00.239366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f80556-c0e9-440a-92d6-792f12377a82 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.240185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f80556-c0e9-440a-92d6-792f12377a82 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.240185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f80556-c0e9-440a-92d6-792f12377a82 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.240185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 136/276] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:00.245620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f80556-c0e9-440a-92d6-792f12377a82 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.247438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:00.293760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 141/277] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:00.295115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.295407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.295671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.317770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:00.320429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:00.405017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f80556-c0e9-440a-92d6-792f12377a82 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.408091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 142/278] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:00.441074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.447767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.448401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.448676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.448943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.449609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:00.449909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:00.451296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:00.460396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 137/279] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:00.466074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.479511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:00.573173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820dcd-922d-4f62-b331-e86dfc36f537 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.576057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:00.596806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d0ea9-e2fa-47b3-9ae4-9538fd791872 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.617207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.617846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.618116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.618386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.619592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:00.621095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:00.627578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 143/280] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:00.627578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.636793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.637495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.637767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.638032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] PATH_INFO: `/v3/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.639202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] RBAC: Authorizing `identity:get_user(user_id=09ee35e6dd2a4d89bb0fd9db86e1f2d7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:00.640695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:00.647585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 138/281] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:00.647585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.730457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:00.735540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401179d-1fd0-4934-a177-2c2fae5bf742 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.752210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0b85b-dbf9-458b-ba03-0906aed84864 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.769784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.770409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.770680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.770945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.781014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:00.786151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.786751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:00.786848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:00.787945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:00.788863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:00.789660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.789660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.789660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:00.789841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 144/282] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:00.789841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.794145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:00.815106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:00.827775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 139/283] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:31:00 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:00.827775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.908011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936eb17-d720-4502-9c66-7c4377c93f51 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.941970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd90a0e-e622-4d30-b16c-9a5bb8dd49bb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:00.945236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.945918 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.946192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.946458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.947631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:00.949137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:00.950845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 145/284] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:00.951633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:00.994026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:00.994583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:00.994856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:00.995121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:00.998001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:01.010654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:01.016658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:01.017549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:01.018351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 140/285] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:01.018992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:01.072527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:01.085671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035c1e-4250-4462-9db3-86eec7b19872 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.133256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.134271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.134540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.134808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.135476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:01.135798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.137168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:01.145835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632dded-4402-494f-ae1f-54306d5ca765 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.151577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 146/286] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:01.152526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:01.191736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.192574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.192946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.193311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7/roles/bc847504306847498ec5148b7e5db940` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.195665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] RBAC: Authorizing `identity:create_grant(project_id=e22dee573a0047c6958f6c34395e3b67, user_id=09ee35e6dd2a4d89bb0fd9db86e1f2d7, role_id=bc847504306847498ec5148b7e5db940)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.197421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:01.215396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 141/287] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:31:01 2022] PUT /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7/roles/bc847504306847498ec5148b7e5db940 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:01.216574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:01.284626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32432db-6af9-4897-80b1-df839e85e350 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.285680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:01.353569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.354195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.354465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.354730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.355414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:01.356393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.358764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:01.376202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25c135-ee74-4112-8f16-8ed6e81cf689 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.382288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 147/288] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:01.388085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:01.423395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:01.449286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.450365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.450910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.451182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] PATH_INFO: `/v3/users/c221c2a212184a4d8439c5113772adc7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.471889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] RBAC: Authorizing `identity:get_user(user_id=c221c2a212184a4d8439c5113772adc7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.473505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:01.475184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 142/289] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:01 2022] GET /identity/v3/users/c221c2a212184a4d8439c5113772adc7 => generated 293 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:01.481536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:01.526472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157e4cf-0ca2-4908-9740-81d7c349e0c3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.562195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.562800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.563573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.563893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] PATH_INFO: `/v3/users/146d20c1436542f8908f05efc2452ff1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.565113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] RBAC: Authorizing `identity:get_user(user_id=146d20c1436542f8908f05efc2452ff1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.566550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:01.568335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 148/290] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:01 2022] GET /identity/v3/users/146d20c1436542f8908f05efc2452ff1 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:01.573976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:01.640312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d3c85-5225-48ca-a4b0-cb356341e270 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.655607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:01.678556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.679269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.679606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.679993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.680936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, user.id=c221c2a212184a4d8439c5113772adc7, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:01.681363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.682950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:01.689461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 143/291] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Oct 28 09:31:01 2022] GET /identity/v3/role_assignments?user.id=c221c2a212184a4d8439c5113772adc7&scope.system=all&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:01.763120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:01.792273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29df420-beb5-4c4e-8943-f8a084463818 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:01.848309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:01.849297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:01.849782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:01.850254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:01.937600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:01.940008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:02.065219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:02.088549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 149/292] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:31:02.095125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:02.299625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f4fd2-81fb-4419-884a-992bba261de8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:02.366186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344b143-a237-4298-94f3-68fddc7b488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:02.366944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b143-a237-4298-94f3-68fddc7b488d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:02.367223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b143-a237-4298-94f3-68fddc7b488d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:02.367494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344b143-a237-4298-94f3-68fddc7b488d None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:02.368883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b143-a237-4298-94f3-68fddc7b488d None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:02.370420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344b143-a237-4298-94f3-68fddc7b488d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:02.373583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 144/293] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:02 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:02.380090 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:02.601440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b19ae-99d6-4907-a8ed-b1ca92b15653 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:02.678469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:02.679491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:02.679821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:02.680276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:02.681658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=146d20c1436542f8908f05efc2452ff1) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:02.682375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:02.685039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff534e3-d0fe-4f45-a40a-2417b0baa659 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:02.703357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 150/294] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:02 2022] GET /identity/v3/role_assignments?user.id=146d20c1436542f8908f05efc2452ff1&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:03.483287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058d17c-d23d-46fd-9ef7-300277883a9e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:03.483819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058d17c-d23d-46fd-9ef7-300277883a9e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:03.483819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058d17c-d23d-46fd-9ef7-300277883a9e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:03.486103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 145/295] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:03.492033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:03.492314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:03.492571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:03.496778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13f13-f4ad-46aa-a5b6-9256708d5713 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:03.497051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13f13-f4ad-46aa-a5b6-9256708d5713 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:03.497314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13f13-f4ad-46aa-a5b6-9256708d5713 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:03.498818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 146/296] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:03.504902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:03.505476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:03.505947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:03.515920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:03.517124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:03.526190 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:03.527845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:03.564780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:03.589468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:03.590668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 151/297] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:03.609074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 147/298] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:03.866027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:03.887928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:03.995235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a1121-8eee-4288-86cc-5e60fb25dbbd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.009105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b276b2a-da2f-4b80-b9bc-1c403ab67257 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.032159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce1a18-704b-4556-8a56-7ae44a8459f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.032800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce1a18-704b-4556-8a56-7ae44a8459f5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.033069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce1a18-704b-4556-8a56-7ae44a8459f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.033334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce1a18-704b-4556-8a56-7ae44a8459f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.043386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce1a18-704b-4556-8a56-7ae44a8459f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-53ce1a18-704b-4556-8a56-7ae44a8459f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:04.048344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:04.049277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:04.050086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 152/299] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:04.050747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb1e82-93c2-42d9-9568-a8fcd4ccf07c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.051398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb1e82-93c2-42d9-9568-a8fcd4ccf07c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.051398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb1e82-93c2-42d9-9568-a8fcd4ccf07c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.052432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 153/300] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:04.055110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efadd42e-5c68-47d4-bcb4-cae0afda84e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.055783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadd42e-5c68-47d4-bcb4-cae0afda84e3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.056052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadd42e-5c68-47d4-bcb4-cae0afda84e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.056313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efadd42e-5c68-47d4-bcb4-cae0afda84e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.058982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb1e82-93c2-42d9-9568-a8fcd4ccf07c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.060576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadd42e-5c68-47d4-bcb4-cae0afda84e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:04.063409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efadd42e-5c68-47d4-bcb4-cae0afda84e3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:04.065362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 148/301] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:04.066351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.066627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.066892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.084963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:04.086122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:04.108382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:04.123986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:04.159332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 149/302] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:04.160183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.193189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb1e82-93c2-42d9-9568-a8fcd4ccf07c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.240457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358757-9390-4598-ab21-830591436419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.241323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358757-9390-4598-ab21-830591436419 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.241823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358757-9390-4598-ab21-830591436419 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.242254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2358757-9390-4598-ab21-830591436419 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.243123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358757-9390-4598-ab21-830591436419 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:04.243575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358757-9390-4598-ab21-830591436419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:04.245208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2358757-9390-4598-ab21-830591436419 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:04.255314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 154/303] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:04.260603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358757-9390-4598-ab21-830591436419 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.277657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c4621-2c6e-40bb-bbb1-c0d2252b4ca2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.315262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.316133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.316544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.317046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] PATH_INFO: `/v3/groups` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.317914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:04.319514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:04.344070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 150/304] 10.222.0.52 () {60 vars in 1134 bytes} [Fri Oct 28 09:31:04 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:04.370765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:04.377013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2358757-9390-4598-ab21-830591436419 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.414947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.415603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.415945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.416221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] PATH_INFO: `/v3/users/c0ded63b53b94ea5ae9cef9a6b767edd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.432210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=c0ded63b53b94ea5ae9cef9a6b767edd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:04.433881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:04.435628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 155/305] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:04 2022] GET /identity/v3/users/c0ded63b53b94ea5ae9cef9a6b767edd => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:04.442384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.458683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.488407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:04.593292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc633f-4347-43ea-981a-bd08b13e234e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.606642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70fee7-39ed-4b87-b72b-362c7f200ec8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.636247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.637155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.637604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.638041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.638928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] RBAC: Adding query filter params (role.id=bc847504306847498ec5148b7e5db940, user.id=c0ded63b53b94ea5ae9cef9a6b767edd, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:04.639413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:04.641120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e2697-b50f-4859-87f1-2524a4dc5b7b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:04.646636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 151/306] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Oct 28 09:31:04 2022] GET /identity/v3/role_assignments?user.id=c0ded63b53b94ea5ae9cef9a6b767edd&scope.system=all&role.id=bc847504306847498ec5148b7e5db940 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:04.653244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a716-cffd-4316-9882-9936a8996073 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.653798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a716-cffd-4316-9882-9936a8996073 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.654072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2a716-cffd-4316-9882-9936a8996073 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.655413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 152/307] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:04.659333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.660315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.660769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.661323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.661679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.662057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.662463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.675770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:04.681350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:04.682298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:04.683101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:04.685180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 156/308] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:04.690590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:04.690590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:04.693000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.721982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:04.791854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 153/309] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:04.836382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cec4f1-5427-4517-bc5f-f109d2caf25c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:04.872242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:04.872880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:04.873123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:04.873392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:04.874052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:04.874353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:04.875902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:04.884601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 157/310] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:04.890652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:04.919629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:04.997605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb6d6c-de02-4919-9284-c65bc71b9676 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.033776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.034781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.035257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.035719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] PATH_INFO: `/v3/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.037055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] RBAC: Authorizing `identity:get_user(user_id=09ee35e6dd2a4d89bb0fd9db86e1f2d7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.038832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.041009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 154/311] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:04 2022] GET /identity/v3/users/09ee35e6dd2a4d89bb0fd9db86e1f2d7 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:05.047893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.071456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.142088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:05.170311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3a9c1-56eb-41ba-b517-3328440f0e52 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.193899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:05.205270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dab48-a1f5-4a4f-ba94-2dc717a87650 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.212804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.214254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.214738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.215811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.248839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.250072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.250531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.251167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.269042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:05.274806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:05.275699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:05.276568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:05.280976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 155/312] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:05.282578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.309274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.311013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.400618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2562d7-3ea0-4458-b884-20ee040da9d8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.411854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:05.442325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 158/313] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:05.447894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.449939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.450782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.451301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.451806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.452699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:05.453826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.455440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.464793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 156/314] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:05.470234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.494444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:05.577724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676c83a-203e-470f-bedc-3673825536cc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.590769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d57f9f-5da9-42a5-b982-c1853cef38f8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.623196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.624238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.624817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.625900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.627295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.628188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.628823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.629097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.629366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] PATH_INFO: `/v3/users/c221c2a212184a4d8439c5113772adc7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.630054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.630523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] RBAC: Authorizing `identity:get_user(user_id=c221c2a212184a4d8439c5113772adc7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.636814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.637556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 159/316] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:05 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:05.637668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 157/316] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:05 2022] GET /identity/v3/users/c221c2a212184a4d8439c5113772adc7 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:05.638452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.641873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:05.732147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:05.746739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:05.761576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00077fd5-096f-48a4-abc8-f1bdf523fa1c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.768404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982678-c433-4658-bbf0-f376985711ed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:05.802734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a47970-aa7b-4c31-8915-a4519d337422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.803782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a47970-aa7b-4c31-8915-a4519d337422 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.804072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a47970-aa7b-4c31-8915-a4519d337422 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.804343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a47970-aa7b-4c31-8915-a4519d337422 None admin] PATH_INFO: `/v3/system/users/c221c2a212184a4d8439c5113772adc7/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.806053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a47970-aa7b-4c31-8915-a4519d337422 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c221c2a212184a4d8439c5113772adc7, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.807499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a47970-aa7b-4c31-8915-a4519d337422 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.813518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:05.814621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:05.815261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:05.815836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:05.816963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] RBAC: Adding query filter params (role.id=bc847504306847498ec5148b7e5db940, scope.project.id=e22dee573a0047c6958f6c34395e3b67, user.id=09ee35e6dd2a4d89bb0fd9db86e1f2d7) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:05.817469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:05.819203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc60afe-236c-41dd-bb32-34a6843395b5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:05.824462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 160/317] 10.222.0.52 () {58 vars in 1334 bytes} [Fri Oct 28 09:31:05 2022] PUT /identity/v3/system/users/c221c2a212184a4d8439c5113772adc7/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:31:05.835770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 158/318] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:05 2022] GET /identity/v3/role_assignments?user.id=09ee35e6dd2a4d89bb0fd9db86e1f2d7&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=bc847504306847498ec5148b7e5db940 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:06.670003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2286b-a218-4f06-b6c9-bfd79e82fec7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:06.670521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2286b-a218-4f06-b6c9-bfd79e82fec7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:06.670521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2286b-a218-4f06-b6c9-bfd79e82fec7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:06.672627 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 161/319] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:06.677875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc2eb4-237d-4f23-b267-8a41d85c07c5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:06.678318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc2eb4-237d-4f23-b267-8a41d85c07c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:06.678411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc2eb4-237d-4f23-b267-8a41d85c07c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:06.697672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bcbc2eb4-237d-4f23-b267-8a41d85c07c5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:06.698874 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bcbc2eb4-237d-4f23-b267-8a41d85c07c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:06.777929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 159/320] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:06.889302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a5c52-6397-42af-8a92-25393bde9f97 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:06.889694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a5c52-6397-42af-8a92-25393bde9f97 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:06.889873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a5c52-6397-42af-8a92-25393bde9f97 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:06.891485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 162/321] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:06.896623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:06.896918 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:06.897181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:06.915579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:06.916791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:06.988549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 160/322] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:07.045424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a5c52-6397-42af-8a92-25393bde9f97 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:07.183241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a5c52-6397-42af-8a92-25393bde9f97 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:07.199592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:07.224316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:07.224949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:07.225229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:07.225494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:07.227453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:07.231776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:07.231776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 163/323] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:07.237958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:07.259565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:07.261048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:07.378367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ac921-50a3-49ee-a20e-8c9c751d8a9c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:07.385284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b44fa4-c984-4c5b-a1ae-19c8e250e07d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:07.417631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:07.419094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:07.419708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:07.420191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] PATH_INFO: `/v3/groups` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:07.421273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:07.423614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:07.435548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:07.436607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:07.436996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:07.437310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:07.451797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 161/324] 10.222.0.52 () {60 vars in 1134 bytes} [Fri Oct 28 09:31:07 2022] POST /identity/v3/groups => generated 226 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:31:07.457631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:07.464196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:07.466017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:07.467595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 164/325] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:07.475121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:07.588968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:07.658215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdacb149-6731-4c5b-98ef-ea30cec83d25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:07.715921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:07.716870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:07.716870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:07.717105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:07.717725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:07.718021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:07.719487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:07.731384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 162/326] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:07.737601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:07.813255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:07.911432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7a06b-b47d-4884-849e-f84f00071ff5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:07.977700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a6b66-fa10-4ffa-8fb7-f23516de484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:07.978785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6b66-fa10-4ffa-8fb7-f23516de484f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:07.979276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6b66-fa10-4ffa-8fb7-f23516de484f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:07.979785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a6b66-fa10-4ffa-8fb7-f23516de484f None admin] PATH_INFO: `/v3/users/c0ded63b53b94ea5ae9cef9a6b767edd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:07.981947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a6b66-fa10-4ffa-8fb7-f23516de484f None admin] RBAC: Authorizing `identity:get_user(user_id=c0ded63b53b94ea5ae9cef9a6b767edd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:07.984775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a6b66-fa10-4ffa-8fb7-f23516de484f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:07.988093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 165/327] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:07 2022] GET /identity/v3/users/c0ded63b53b94ea5ae9cef9a6b767edd => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:07.995790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:08.076851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de72f-91c6-4bd8-a50a-9fb61b29d7f0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:08.077770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de72f-91c6-4bd8-a50a-9fb61b29d7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:08.078614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de72f-91c6-4bd8-a50a-9fb61b29d7f0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:08.082679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 166/328] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:08.089193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:08.089439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:08.089739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:08.112270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:08.113466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:08.199993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565bf2e-a260-4e9c-bca4-1f3f6f97ed1f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:08.231567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:08.238991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:08.239897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:08.240189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:08.240431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] PATH_INFO: `/v3/system/users/c0ded63b53b94ea5ae9cef9a6b767edd/roles/bc847504306847498ec5148b7e5db940` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:08.242742 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c0ded63b53b94ea5ae9cef9a6b767edd, role_id=bc847504306847498ec5148b7e5db940)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:08.244319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:08.256748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 167/329] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:08.260513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 163/330] 10.222.0.52 () {58 vars in 1334 bytes} [Fri Oct 28 09:31:07 2022] PUT /identity/v3/system/users/c0ded63b53b94ea5ae9cef9a6b767edd/roles/bc847504306847498ec5148b7e5db940 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:08.646116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:08.695339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:08.766036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee34413-17c9-4748-844d-fd92c89b7533 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:08.818475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:08.819165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:08.819438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:08.819708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:08.836331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:08.842966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:08.845223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:08.846727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 168/331] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:08.856107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:09.062987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f10a9-b1b0-4f3a-b357-c30b0a477ac0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:09.152173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:09.153178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:09.153648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:09.154113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:09.155311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:09.155867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:09.158987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:09.175328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 164/332] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:09.180865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:09.394834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:09.401049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccc0fc-e6c1-4df9-8641-ab51bd49159f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:09.468075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578f0fb-f178-4583-b08a-3d5b7ca0ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:09.469181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578f0fb-f178-4583-b08a-3d5b7ca0ae2e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:09.469640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578f0fb-f178-4583-b08a-3d5b7ca0ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:09.470113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578f0fb-f178-4583-b08a-3d5b7ca0ae2e None admin] PATH_INFO: `/v3/users/c221c2a212184a4d8439c5113772adc7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:09.472083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578f0fb-f178-4583-b08a-3d5b7ca0ae2e None admin] RBAC: Authorizing `identity:get_user(user_id=c221c2a212184a4d8439c5113772adc7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:09.474695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578f0fb-f178-4583-b08a-3d5b7ca0ae2e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:09.477920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 169/333] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:09 2022] GET /identity/v3/users/c221c2a212184a4d8439c5113772adc7 => generated 293 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:09.487275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:09.613146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:09.715847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453e513-57ec-402a-ac2c-833389db5c02 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:09.790065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:09.791561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:09.792329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:09.793112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:09.794787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] RBAC: Adding query filter params (role.id=39a47ee6b3f14e6ead39370f270c2f38, user.id=c221c2a212184a4d8439c5113772adc7, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:09.795586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:09.798940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad61674-23b3-4927-9846-0cafa0e3c34a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:09.827115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 165/334] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Oct 28 09:31:09 2022] GET /identity/v3/role_assignments?user.id=c221c2a212184a4d8439c5113772adc7&scope.system=all&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 510 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:12.175165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c11484-3f28-4b30-bbba-c5a2339eea5d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.175695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c11484-3f28-4b30-bbba-c5a2339eea5d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.175695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c11484-3f28-4b30-bbba-c5a2339eea5d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.177759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 170/335] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:12.184083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472581df-8219-484b-847c-9c30298df51f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.185036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472581df-8219-484b-847c-9c30298df51f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.185502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472581df-8219-484b-847c-9c30298df51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.209277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-472581df-8219-484b-847c-9c30298df51f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:12.210871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-472581df-8219-484b-847c-9c30298df51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:12.311420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffacea-22a0-42a1-bb9c-374294842a32 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.311807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffacea-22a0-42a1-bb9c-374294842a32 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.312061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffacea-22a0-42a1-bb9c-374294842a32 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.312795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 166/336] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:12.316026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 171/337] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:12.320691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188bad-4547-4277-898e-0f3a00428512 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.321296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188bad-4547-4277-898e-0f3a00428512 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.321769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b188bad-4547-4277-898e-0f3a00428512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.346197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2b188bad-4547-4277-898e-0f3a00428512 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:12.347834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2b188bad-4547-4277-898e-0f3a00428512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:12.409248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:12.451680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 167/338] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:12.660070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ceeb-06e1-440c-b7d0-0ac3457f879f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.660479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ceeb-06e1-440c-b7d0-0ac3457f879f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.660967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9ceeb-06e1-440c-b7d0-0ac3457f879f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.663909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 172/339] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:12.675758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab1b60-d381-414b-bbdb-b543c1bb49d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.676368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab1b60-d381-414b-bbdb-b543c1bb49d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.676860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab1b60-d381-414b-bbdb-b543c1bb49d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.704362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-28ab1b60-d381-414b-bbdb-b543c1bb49d7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:12.705915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28ab1b60-d381-414b-bbdb-b543c1bb49d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:12.806575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 168/340] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:12.837522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a10b2-1016-4bc6-807b-1e7c088e15b7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.838005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a10b2-1016-4bc6-807b-1e7c088e15b7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.838477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a10b2-1016-4bc6-807b-1e7c088e15b7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.841089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 173/341] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:12.847877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:12.848515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:12.848998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:12.867773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a10b2-1016-4bc6-807b-1e7c088e15b7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:12.874063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:12.875337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:13.006164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 169/342] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:13.160436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:13.189576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a10b2-1016-4bc6-807b-1e7c088e15b7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.244071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:13.252477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:13.253383 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:13.253811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:13.254227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:13.284941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:13.290270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:13.291409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:13.292452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:13.300100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 174/343] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:13.301315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:13.370410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:13.430589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814aa6c9-798a-4c51-86e5-2cb324b97ece None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.503494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dc8d6-047f-4458-84af-e397e0836184 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.515514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:13.517054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:13.517816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:13.518508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:13.551138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:13.565197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:13.565862 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:13.566135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:13.566394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:13.567523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:13.568652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 28 09:31:13.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:13.573284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 170/344] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:13.575147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:13.595788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:13.601765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:13.602908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:13.603950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:13.605972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 175/345] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:13.606966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:13.740366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:13.742641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:13.757633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c0ab9-e10f-4f33-93b8-cb54f690eb8d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.766796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2dbda9-56a2-457c-ba45-9eb46a233146 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.803840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:13.804372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:13.804639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:13.804914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:13.805595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:13.805896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:13.807293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:13.817899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:13.818558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:13.818834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:13.819099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:13.819262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 171/346] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:13.820485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:13.834947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:13.840392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:13.841337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:13.842171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 176/347] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:13.842847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:13.946926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a815b-6294-4d2e-b5f1-ff915031694f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.986908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078aa0c-b343-4701-89c0-a6f0c0868777 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:13.995554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:13.995554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:13.995554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:13.995947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:13.997230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:13.997547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:13.998977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409e76b-66a6-4ead-9303-6b998d8dd7bb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:14.009249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 172/348] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:14.010671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5f29-b10f-42b1-8f32-3be23d2201bb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.010964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5f29-b10f-42b1-8f32-3be23d2201bb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.011232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca5f29-b10f-42b1-8f32-3be23d2201bb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.013068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 173/349] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:14.013422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5f29-b10f-42b1-8f32-3be23d2201bb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:14.037394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:14.038025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.038300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.038559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.039251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:14.039609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:14.041058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21109b-150b-455c-9b0f-9e59b1a28cd4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:14.044206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:14.053103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 177/350] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:14.054749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6279b65-f218-4f3e-aae2-99dd1445a13f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.054966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6279b65-f218-4f3e-aae2-99dd1445a13f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.055241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6279b65-f218-4f3e-aae2-99dd1445a13f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.056742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 178/351] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:14.057400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6279b65-f218-4f3e-aae2-99dd1445a13f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:14.072292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:14.183680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca5f29-b10f-42b1-8f32-3be23d2201bb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:14.228543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6279b65-f218-4f3e-aae2-99dd1445a13f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:14.250427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a49e88-c5ff-414b-901e-1586766c067d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:14.251431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a49e88-c5ff-414b-901e-1586766c067d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.251922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a49e88-c5ff-414b-901e-1586766c067d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.252365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a49e88-c5ff-414b-901e-1586766c067d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.293473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:14.294422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.294861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.295260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.296442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:14.296922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:14.299350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:14.319837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 179/352] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:14.321106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:14.380168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a49e88-c5ff-414b-901e-1586766c067d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:14.384217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a49e88-c5ff-414b-901e-1586766c067d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:14.421578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:14.446407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:14.498942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b9303-2c60-4280-8b26-a96d40688437 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:14.561515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 174/353] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:14.564200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.564683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.565291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.566929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fa078-4067-4e92-bffc-27de70847520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:14.568167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa078-4067-4e92-bffc-27de70847520 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.568920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa078-4067-4e92-bffc-27de70847520 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.569630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fa078-4067-4e92-bffc-27de70847520 None admin] PATH_INFO: `/v3/users/c0ded63b53b94ea5ae9cef9a6b767edd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.571660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa078-4067-4e92-bffc-27de70847520 None admin] RBAC: Authorizing `identity:get_user(user_id=c0ded63b53b94ea5ae9cef9a6b767edd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:14.574010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fa078-4067-4e92-bffc-27de70847520 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:14.577263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 180/354] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:31:14 2022] GET /identity/v3/users/c0ded63b53b94ea5ae9cef9a6b767edd => generated 293 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:14.580027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:14.580492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:14.580953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:14.597992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:14.600083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:14.610290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:14.612325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:14.725975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 181/355] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:14.727458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:14.731858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 175/356] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:14.733643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:14.835057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:14.944434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:14.952711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966097b-870e-493c-bd24-f269bab921a4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:14.957547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640edb6-de6f-4fc9-9190-8a322b564d93 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:15.034903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:15.035960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:15.036519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:15.037077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:15.038389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:15.039335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:15.039774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:15.040259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:15.184117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:15.188489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:15.198474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:15.201123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:15.287829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:15.335711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:15.388018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 176/357] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:15.391850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:15.399651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 182/358] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:15.401438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:15.568668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931308a-a926-4661-96a3-a8c8cfe43bda None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:15.570415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864602eb-aa16-400e-9da1-0744c25def78 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:15.611854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:15.612672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:15.613096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:15.613477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:15.614720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:15.616278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:15.616488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:15.616883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:15.617152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:15.617422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:15.618147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] RBAC: Adding query filter params (role.id=bc847504306847498ec5148b7e5db940, user.id=c0ded63b53b94ea5ae9cef9a6b767edd, scope.system=all) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:15.618453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:15.618624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 177/359] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:15 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:15.619803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:15.619959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:15.639255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 183/360] 10.222.0.52 () {56 vars in 1308 bytes} [Fri Oct 28 09:31:15 2022] GET /identity/v3/role_assignments?user.id=c0ded63b53b94ea5ae9cef9a6b767edd&scope.system=all&role.id=bc847504306847498ec5148b7e5db940 => generated 510 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:15.640471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:15.646618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:15.682914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:15.797359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236dc457-a95c-460a-b050-0f87d192c41a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:15.807339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f8bf1-f8e9-4903-b929-fe59d19a0b0a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:15.854812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:15.855660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:15.856154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:15.856553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:15.859888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:15.860906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:15.861363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:15.861787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:15.870962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:15.877191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:15.879295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:15.881061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:31:15.884517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:15.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:15.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:15.892158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 178/361] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:15.895237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 184/362] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:31:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:15.895237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:15.896069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.025361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:16.034334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:16.060356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9e3e-d3de-4f18-a77b-c29b413ee256 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.068666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee394806-31e0-424b-bde7-7682d8019cb8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.119540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.120590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.120952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.121623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.123159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.123879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.125567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.126079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.126222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.126359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.128040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.130308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 185/363] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:15 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.130470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.131552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.133401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 179/364] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:15 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.142160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.289053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0e8e-5f75-456c-9e9b-dd89be4816e5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.300659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d00517-9e12-443e-877f-b26ad6fe955f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.337784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.338774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.339215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.339652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.347302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.348311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.348748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.349195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.349792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.350279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:16.350757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.352309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.353074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.361578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 186/365] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.361578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.369972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 180/366] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.371065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.381881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:16.383031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:16.481046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66cdbb4-059a-42b8-8ba1-0b4931f695b8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.506576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1ef82-fe87-4acf-ac6f-9698d45b5094 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.524100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.524841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.525237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.525590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.526406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:16.526796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.528531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.537970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 187/367] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.539325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.550478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.551243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.551594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.551967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.553201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.554843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.556645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 181/368] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.557977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.659901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:16.668051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:16.685820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826ee8f-87cc-4652-8492-5a9f5212330f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.704231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee8398-70b7-4c2e-9809-3948c0d95f04 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.734901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.735736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.736160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.736565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.738025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.739688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.741676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 188/369] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.743038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.752492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.753432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.753900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.754220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.755101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=e22dee573a0047c6958f6c34395e3b67) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:16.755451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:16.757199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:16.766031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 182/370] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:16.767401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:16.909346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5a31-642b-4e27-8f66-f6f3549bfa45 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.911311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d131da4-0388-4f95-8745-a9a275b1736e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:16.966668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.966926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:16.967643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.967862 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:16.968655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.968843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:16.969091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.969245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:16.986146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:17.000392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:17.075935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:17.078555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:17.102221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:17.104986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:17.243797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 183/371] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:17.243797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:17.272288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 189/372] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:17.273708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:17.321046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:17.341881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:17.419723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383d9f-c8cc-4315-84d3-8601fca6918c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:17.440220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15b590-e482-4932-aef3-15e348b1720d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:17.482899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:17.483682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:17.483984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:17.484353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:17.485111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:17.485515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:17.487015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:17.487389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:17.488453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:17.488982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:17.489416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:17.490806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:17.491359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:17.494355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:17.500010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 184/373] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:17 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:17.501314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:17.524124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 190/374] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:17 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:17.529743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:17.803599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3872811-893c-483c-bf38-bd7b48a5efb8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:17.807053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:17.826478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12006ab7-cf92-42a1-b33d-a5bfb10edba1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:17.863889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:17.906303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:17.908104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:17.908888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:17.909674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:17.912081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:17.915036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:17.923823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 185/375] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:17 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:17.930144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:17.953846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:17.954930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:17.955410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:17.956015 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] PATH_INFO: `/v3/projects/afd3325807f94fa3b6a7d418f71aaa51` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:17.958105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] RBAC: Authorizing `identity:get_project(project_id=afd3325807f94fa3b6a7d418f71aaa51)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:17.960769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:17.964141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 191/376] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:17 2022] GET /identity/v3/projects/afd3325807f94fa3b6a7d418f71aaa51 => generated 344 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:17.976972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:18.199253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67736a-18b2-4127-91ef-47e4ea789ed3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:18.273200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ea2a6-f56e-4c46-88ac-b7765df6b01f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:18.274705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:18.276155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:18.276908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:18.277642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:18.285122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:18.287930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:18.291453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 186/377] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:17 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:18.305560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:18.310985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:18.311671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:18.312070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:18.312965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] PATH_INFO: `/v3/groups/93d04f042327486d844c6d8f81c53a95` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:18.325939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] RBAC: Authorizing `identity:get_group(group_id=93d04f042327486d844c6d8f81c53a95)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:18.327418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:18.330470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 192/378] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:17 2022] GET /identity/v3/groups/93d04f042327486d844c6d8f81c53a95 => generated 229 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:18.339071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:18.397095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:18.427605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:18.508355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a021a-71ba-4553-aa6f-2c46b5217233 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:18.574341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:18.578530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab865aa-cb2d-4308-879d-b9493f783184 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:18.580058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:18.580843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:18.581638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:18.583323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=e22dee573a0047c6958f6c34395e3b67) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:18.587330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:18.590588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2929f-d8ad-4c68-a182-eaa42ad6328a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:18.623971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 187/379] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:18 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 260 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:18.688809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:18.690382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:18.691165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:18.691976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:18.693753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] RBAC: Adding query filter params (group.id=93d04f042327486d844c6d8f81c53a95, role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=afd3325807f94fa3b6a7d418f71aaa51) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:18.694656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:18.698003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd628de-c696-4118-9a09-64a59df68b6c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:18.719961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 193/380] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:18 2022] GET /identity/v3/role_assignments?group.id=93d04f042327486d844c6d8f81c53a95&scope.project.id=afd3325807f94fa3b6a7d418f71aaa51&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 260 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:19.756636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd739932-8a24-4158-8c30-bd7d51e0d555 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:19.758214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd739932-8a24-4158-8c30-bd7d51e0d555 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:19.758999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd739932-8a24-4158-8c30-bd7d51e0d555 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:19.762399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 188/381] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:19.770859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6061f80-57c3-4b15-978d-806617db9423 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:19.772077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6061f80-57c3-4b15-978d-806617db9423 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:19.772704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6061f80-57c3-4b15-978d-806617db9423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:19.811722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6061f80-57c3-4b15-978d-806617db9423 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:19.814186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6061f80-57c3-4b15-978d-806617db9423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:19.991924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 194/382] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:20.494041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193609fc-6a4d-40e4-a339-cecd95a2f998 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:20.495064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193609fc-6a4d-40e4-a339-cecd95a2f998 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:20.495890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193609fc-6a4d-40e4-a339-cecd95a2f998 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:20.499567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 189/383] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:20.509074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:20.509923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:20.510581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:20.542483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193609fc-6a4d-40e4-a339-cecd95a2f998 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:20.548663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:20.549906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:20.604170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:20.672200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 195/384] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:20.724548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193609fc-6a4d-40e4-a339-cecd95a2f998 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:20.726563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:20.781054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:20.781673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:20.781962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:20.782239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:20.799810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:20.804862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:20.805756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:20.806563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 190/385] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:20.813395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.041239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7f56fc-65b9-4eab-be6f-389f26113b6f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:21.080104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.115724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:21.116760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.117443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.117927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.119137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:21.119661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:21.122296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:21.153979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 196/386] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:21.164164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.231251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:21.381871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774f306-8651-4732-a239-d6befbffc90f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:21.383069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcaf03-844d-48ae-b8a5-d607d9ae15e2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:21.428256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeb2f4-a2df-4582-8998-76966bbb354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:21.428903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb2f4-a2df-4582-8998-76966bbb354b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.429181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb2f4-a2df-4582-8998-76966bbb354b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.429452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb2f4-a2df-4582-8998-76966bbb354b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.444446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbeb2f4-a2df-4582-8998-76966bbb354b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7fbeb2f4-a2df-4582-8998-76966bbb354b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:21.450591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:21.452735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:21.454531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:21.457207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 191/387] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:21.458696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6920a1-adda-4154-867f-b94adc7a5f6a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.458988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6920a1-adda-4154-867f-b94adc7a5f6a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.459259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6920a1-adda-4154-867f-b94adc7a5f6a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.460686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 192/388] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:21.461710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6920a1-adda-4154-867f-b94adc7a5f6a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.464966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b21d5f0-5437-44ee-b07e-915922b66c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:21.466359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21d5f0-5437-44ee-b07e-915922b66c2d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.467144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21d5f0-5437-44ee-b07e-915922b66c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.468446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b21d5f0-5437-44ee-b07e-915922b66c2d None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.472146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21d5f0-5437-44ee-b07e-915922b66c2d None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:21.473718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b21d5f0-5437-44ee-b07e-915922b66c2d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:21.475505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 197/389] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:21 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:21.476819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.477088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.477357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.513803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:21.517427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:21.525653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:21.630281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6920a1-adda-4154-867f-b94adc7a5f6a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:21.653114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 198/390] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:21.655018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.692553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:21.693189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.693465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.693734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.694422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:21.694732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:21.696184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:21.709407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:21.729616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 193/391] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:21.738988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.881715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:21.903058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb07e14-d27b-4a11-bca9-17b2d940d228 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:21.905841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6f049-43d7-4e3e-8850-3cebe7ed7506 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:21.944808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123d541-772f-498b-8a02-733ade4e0cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:21.945664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d541-772f-498b-8a02-733ade4e0cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.946012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d541-772f-498b-8a02-733ade4e0cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.946379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123d541-772f-498b-8a02-733ade4e0cb2 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.947767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d541-772f-498b-8a02-733ade4e0cb2 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:21.949452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123d541-772f-498b-8a02-733ade4e0cb2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:21.951675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 194/392] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:21 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:21.953139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4a1bd-3dd9-4d62-af78-4741de6d4bdc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.953483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4a1bd-3dd9-4d62-af78-4741de6d4bdc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.953823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4a1bd-3dd9-4d62-af78-4741de6d4bdc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.956359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 195/393] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:21.960766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4a1bd-3dd9-4d62-af78-4741de6d4bdc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:21.964740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58f2cf-d97c-414f-a0bb-7529731f489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:21.965419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f2cf-d97c-414f-a0bb-7529731f489d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.965699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f2cf-d97c-414f-a0bb-7529731f489d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.965970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f2cf-d97c-414f-a0bb-7529731f489d None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:21.969006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f2cf-d97c-414f-a0bb-7529731f489d None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:21.970482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f2cf-d97c-414f-a0bb-7529731f489d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:21.972496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 199/394] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:21 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:21.973499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:21.973778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:21.974047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.003483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:22.005307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:22.092372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4a1bd-3dd9-4d62-af78-4741de6d4bdc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:22.115775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 200/395] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:22.115775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.145361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:22.180396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:22.181598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.181877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.182149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.184276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:22.185731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:22.187624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 196/396] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:21 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:22.188562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.218953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:22.266249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8235abf-4898-4b53-9cd4-0e5729a6910f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:22.312656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:22.314041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.314473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.314889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67/groups/2865aac734d549e8886139f68a2004d9/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.317933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] RBAC: Authorizing `identity:create_grant(project_id=e22dee573a0047c6958f6c34395e3b67, group_id=2865aac734d549e8886139f68a2004d9, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:22.320094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:22.333869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23e304-852b-4f62-b5db-2b0ed6f23cbf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:22.344631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 201/397] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Oct 28 09:31:22 2022] PUT /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67/groups/2865aac734d549e8886139f68a2004d9/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:22.348089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.410821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:22.411652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.412044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.412407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.446166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:22.452644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:22.453615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:22.454523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:22.460393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 197/398] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:22.469113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.478392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:22.499845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82824be0-aca2-4fa9-be6f-043653429706 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:22.545935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:22.546857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.548073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.548533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/groups/2865aac734d549e8886139f68a2004d9/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.551240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7b19705b76444eba9e2a0476a6ff3e, group_id=2865aac734d549e8886139f68a2004d9, role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:22.553370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:22.570311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 202/399] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Oct 28 09:31:22 2022] PUT /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/groups/2865aac734d549e8886139f68a2004d9/roles/7682e93d1a8e439ba530f0911772aa20 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:22.613227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.703523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:22.736834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d008b2-d960-4a33-96d0-7a3b05fbc9d6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:22.754919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbea26-7a78-42ae-a5ef-8f2774a90f0c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:22.798268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:22.799259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.799678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.800133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.801278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:22.801751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:22.803683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abd1e5-a782-49df-bb11-38c2ee214d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:22.804114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77ad260-b584-4db2-97e9-a9e1e7c82d33 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:22.804511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abd1e5-a782-49df-bb11-38c2ee214d81 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.804858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abd1e5-a782-49df-bb11-38c2ee214d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.805195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abd1e5-a782-49df-bb11-38c2ee214d81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.819355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 198/400] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:22.820658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abd1e5-a782-49df-bb11-38c2ee214d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:22.821504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f83ba9-001c-464c-a1a4-72447869c57a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.821946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f83ba9-001c-464c-a1a4-72447869c57a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.822371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f83ba9-001c-464c-a1a4-72447869c57a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.825367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 199/401] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e6abd1e5-a782-49df-bb11-38c2ee214d81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:22.826835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:22.828323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:22.829147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 203/402] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:22.831953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f83ba9-001c-464c-a1a4-72447869c57a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.839975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:22.840501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:22.840774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:22.854795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:22.862032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:22.864295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:22.952926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 204/403] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:22.953901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:22.966905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f83ba9-001c-464c-a1a4-72447869c57a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:23.026844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:23.027495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:23.027790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:23.028726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:23.030802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:23.035865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:23.040339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 200/404] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:22 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:23.043059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:23.076325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:23.153122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1059ae-8b39-47fa-bafa-db58efa7f40c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:23.198076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:23.198723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:23.198990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:23.199264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:23.200010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:23.200312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:23.202317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:23.205356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:23.213240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 205/405] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:23.219944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:23.318301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0787b-96d9-422d-b7c5-30e5aa9bc4a9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:23.392913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785b067-5d31-4386-8175-77513b58f09b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:23.401510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:23.403007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:23.403693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:23.404458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:23.413138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:23.416242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:23.419783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 201/406] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:23 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:23.433168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:23.459453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:23.460926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:23.461905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:23.462648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:23.464903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:23.472547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:23.477428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 206/407] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:23 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:23.482741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:23.597302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:23.604908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:23.641267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d399c53-b0c1-4efb-a781-044c335ba17f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:23.678333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ab0d3-fb3b-42f0-86b9-14388456b37a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:23.694501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:23.695216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:23.695490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:23.695795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/groups/2865aac734d549e8886139f68a2004d9/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:23.697984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] RBAC: Authorizing `identity:create_grant(project_id=fc7b19705b76444eba9e2a0476a6ff3e, group_id=2865aac734d549e8886139f68a2004d9, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:23.700232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:23.730242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 202/408] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Oct 28 09:31:23 2022] PUT /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e/groups/2865aac734d549e8886139f68a2004d9/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:23.735969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:23.782883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:23.784145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:23.784520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:23.785119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:23.788301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:23.791366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:23.795097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 207/409] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:23 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:23.802304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:23.981689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8575dc0-e5d0-4170-9b23-2aef1585ddce None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:24.015340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925b7aa-6c03-4047-becf-0d2df032790b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:24.047850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:24.049617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:24.050086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:24.050556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:24.066914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:24.070076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:24.074073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:24.074655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:24.075121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67/groups/2865aac734d549e8886139f68a2004d9/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:24.077678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:24.079238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] RBAC: Authorizing `identity:create_grant(project_id=e22dee573a0047c6958f6c34395e3b67, group_id=2865aac734d549e8886139f68a2004d9, role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:24.084430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:24.085346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:31:24.086138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:24.088866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:24.091554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:24.095994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 203/410] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:31:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:24.108962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:24.127929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 208/411] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Oct 28 09:31:23 2022] PUT /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67/groups/2865aac734d549e8886139f68a2004d9/roles/7682e93d1a8e439ba530f0911772aa20 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:24.283699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25819-44a1-4242-bef2-47478d0511dd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:24.330227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:24.330864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:24.331144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:24.331494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:24.332227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:24.332530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:24.333965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:24.350308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 204/412] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:24.356679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:24.558582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198fcae-b0ba-46e4-9be0-379c05af5e81 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:24.636949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:24.638053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:24.638529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:24.638970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] PATH_INFO: `/v3/groups/93d04f042327486d844c6d8f81c53a95` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:24.645102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] RBAC: Authorizing `identity:get_group(group_id=93d04f042327486d844c6d8f81c53a95)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:24.647778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:24.651128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 209/413] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:24 2022] GET /identity/v3/groups/93d04f042327486d844c6d8f81c53a95 => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:24.656422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:24.751974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:24.814580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa0945-1351-4d7f-8f00-d0a868d25a7e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:24.865057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f09087-a219-4f56-98c2-9ce7665515b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:24.866227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09087-a219-4f56-98c2-9ce7665515b8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:24.866655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09087-a219-4f56-98c2-9ce7665515b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:24.867091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f09087-a219-4f56-98c2-9ce7665515b8 None admin] PATH_INFO: `/v3/projects/afd3325807f94fa3b6a7d418f71aaa51` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:24.872493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09087-a219-4f56-98c2-9ce7665515b8 None admin] RBAC: Authorizing `identity:get_project(project_id=afd3325807f94fa3b6a7d418f71aaa51)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:24.875305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f09087-a219-4f56-98c2-9ce7665515b8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:24.877608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 205/414] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:24 2022] GET /identity/v3/projects/afd3325807f94fa3b6a7d418f71aaa51 => generated 344 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:24.884134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:25.047663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:25.131505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59503af0-4ae8-4827-8c7d-3debab7384d9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:25.227148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ef658-0b4c-4888-a314-9baebc8aed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:25.228304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ef658-0b4c-4888-a314-9baebc8aed36 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:25.228785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ef658-0b4c-4888-a314-9baebc8aed36 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:25.229246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ef658-0b4c-4888-a314-9baebc8aed36 None admin] PATH_INFO: `/v3/projects/afd3325807f94fa3b6a7d418f71aaa51/groups/93d04f042327486d844c6d8f81c53a95/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:25.232523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ef658-0b4c-4888-a314-9baebc8aed36 None admin] RBAC: Authorizing `identity:create_grant(project_id=afd3325807f94fa3b6a7d418f71aaa51, group_id=93d04f042327486d844c6d8f81c53a95, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:25.235843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ef658-0b4c-4888-a314-9baebc8aed36 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:25.276978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 210/415] 10.222.0.52 () {58 vars in 1442 bytes} [Fri Oct 28 09:31:24 2022] PUT /identity/v3/projects/afd3325807f94fa3b6a7d418f71aaa51/groups/93d04f042327486d844c6d8f81c53a95/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:25.660611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7045-4dd7-4036-aee6-945af1c01d40 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:25.661167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7045-4dd7-4036-aee6-945af1c01d40 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:25.661577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a7045-4dd7-4036-aee6-945af1c01d40 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:25.664491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 206/416] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:25.674800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10fb8-c8aa-4b12-ac0c-9d5a07069ed1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:25.675291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10fb8-c8aa-4b12-ac0c-9d5a07069ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:25.675800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee10fb8-c8aa-4b12-ac0c-9d5a07069ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:25.707925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1ee10fb8-c8aa-4b12-ac0c-9d5a07069ed1 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:25.709909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1ee10fb8-c8aa-4b12-ac0c-9d5a07069ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:25.867312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 211/417] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:26.332201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724bfe8-e308-470c-9635-4828bd7be00e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:26.332648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724bfe8-e308-470c-9635-4828bd7be00e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:26.333089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724bfe8-e308-470c-9635-4828bd7be00e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:26.335913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 207/418] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:26.345011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e12b2c-6488-4f30-8d37-62b360b8dea0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:26.345625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e12b2c-6488-4f30-8d37-62b360b8dea0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:26.345948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e12b2c-6488-4f30-8d37-62b360b8dea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:26.383632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-36e12b2c-6488-4f30-8d37-62b360b8dea0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:26.385675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-36e12b2c-6488-4f30-8d37-62b360b8dea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:26.401205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724bfe8-e308-470c-9635-4828bd7be00e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:26.502401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:26.581455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 212/419] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:26.583100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b58e9-9c3b-4b9a-b19f-ff979fb94cc6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:26.583623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b58e9-9c3b-4b9a-b19f-ff979fb94cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:26.584207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b58e9-9c3b-4b9a-b19f-ff979fb94cc6 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:26.586543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 213/420] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:26.592939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c795d-6323-457b-8f17-654a97b19313 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:26.593622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c795d-6323-457b-8f17-654a97b19313 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:26.594124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c795d-6323-457b-8f17-654a97b19313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:26.627115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-165c795d-6323-457b-8f17-654a97b19313 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:26.629553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-165c795d-6323-457b-8f17-654a97b19313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:26.675370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724bfe8-e308-470c-9635-4828bd7be00e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:26.713698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 214/421] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:26.772003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:26.789221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:26.790310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:26.790863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:26.791375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:26.816938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:26.831445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:26.832434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:26.833245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:26.834439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 208/422] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:26.839841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c795d-6323-457b-8f17-654a97b19313 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.006397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c795d-6323-457b-8f17-654a97b19313 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:27.021658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:27.073208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:27.074335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.074827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.075311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.080731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:27.081284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:27.083978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:27.088642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.097560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 215/423] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:27.108804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.236616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ad76e-9d5a-47b2-8626-c68c5ed8187a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:27.277557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:27.278088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.278362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.278635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.292730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:27.297893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:27.299617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 209/424] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:27.301198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.315768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de4827-79ca-43fc-8638-ea7a8bc29489 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:27.360020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:27.361219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.361219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.361219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.417371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:27.425708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:27.453570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:27.455490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:27.528496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f8d42-1d36-44f7-ae6c-6906e06c5094 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:27.573585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 216/425] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:27.574755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.593959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:27.594949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.595394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.595855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.615409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:27.625250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:27.626354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:27.627315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:27.634923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 210/426] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:27.636515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.669639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:27.701863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dab3b-b082-441c-bfaf-edeb6a3091a7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:27.755693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:27.756373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.756653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.756919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.757596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] RBAC: Adding query filter params (name=member) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:27.757892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:27.759293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:27.772282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 217/427] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:27.775188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.807197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ab9ab-26d7-48b8-91e6-6192c305db92 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:27.825185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:27.872874 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9b487-2c31-4f80-ae7a-8b4be982e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:27.874668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9b487-2c31-4f80-ae7a-8b4be982e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.875145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9b487-2c31-4f80-ae7a-8b4be982e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.875617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9b487-2c31-4f80-ae7a-8b4be982e7f2 None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.877750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9b487-2c31-4f80-ae7a-8b4be982e7f2 None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:27.880478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9b487-2c31-4f80-ae7a-8b4be982e7f2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:27.883645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 211/428] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:27.885920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aecd0-67df-4e75-8f31-0936ee92b532 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:27.886341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aecd0-67df-4e75-8f31-0936ee92b532 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:27.886788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408aecd0-67df-4e75-8f31-0936ee92b532 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:27.889411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 212/429] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:27.893556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aecd0-67df-4e75-8f31-0936ee92b532 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:27.972421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c7d67-8653-4dc2-8cef-d046c9ca4006 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:28.037158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:28.038044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.038483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.038923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.039885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:28.040282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:28.042033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:28.045942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:28.055212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 218/430] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:28.056821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:28.065517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408aecd0-67df-4e75-8f31-0936ee92b532 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:28.127877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:28.128836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.129265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.129725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.205460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:28.246907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2669dab-9dd8-41ab-b85e-70e2a0bc051e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:28.249324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:28.251575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:28.309116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19ac43-e623-47ea-9ba9-367a5a19fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:28.310134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19ac43-e623-47ea-9ba9-367a5a19fbba None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.310572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19ac43-e623-47ea-9ba9-367a5a19fbba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.311049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19ac43-e623-47ea-9ba9-367a5a19fbba None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.312956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19ac43-e623-47ea-9ba9-367a5a19fbba None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:28.315493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19ac43-e623-47ea-9ba9-367a5a19fbba None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:28.318219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 219/431] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:28 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:28.320269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.320741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.321223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.345963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:28.347847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:28.435633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:28.447801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 213/432] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:28.447801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:28.455012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 220/433] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:28.467239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:28.591050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:28.614527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a9bed-fdec-463e-ba36-46c4fa034938 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:28.620193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593bf58-f1af-442c-adf2-a04943acb1e3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:28.660846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:28.661792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.662207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.662621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.669089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:28.670065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.670542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.670975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.672266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=e22dee573a0047c6958f6c34395e3b67) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:28.672761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:28.675229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:28.695783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 221/434] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:28 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:28.695783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:28.809689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:28.813774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:28.815315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:28.901831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235fd83-cc78-4775-918c-33aefa06f931 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:28.966680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd5ee2-40b8-4eb7-a26c-2e03acd9a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:28.967405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd5ee2-40b8-4eb7-a26c-2e03acd9a92c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.967676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd5ee2-40b8-4eb7-a26c-2e03acd9a92c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.967977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd5ee2-40b8-4eb7-a26c-2e03acd9a92c None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.969175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd5ee2-40b8-4eb7-a26c-2e03acd9a92c None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:28.970644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd5ee2-40b8-4eb7-a26c-2e03acd9a92c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:28.972399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 222/435] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:28 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:28.977395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd5d3e-7b22-4f53-b3a8-8c8d9e594fd1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:28.977395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd5d3e-7b22-4f53-b3a8-8c8d9e594fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:28.977629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:28.977704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd5d3e-7b22-4f53-b3a8-8c8d9e594fd1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:28.977704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 223/436] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:28.979032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd5d3e-7b22-4f53-b3a8-8c8d9e594fd1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.020141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 214/437] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:29.021425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.110305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd5d3e-7b22-4f53-b3a8-8c8d9e594fd1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.143634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50300a9d-75cf-452f-b74d-d8777308c187 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.150745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:29.162412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04461d34-7179-4bfd-9a39-714218bce37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.163300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461d34-7179-4bfd-9a39-714218bce37a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.163722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461d34-7179-4bfd-9a39-714218bce37a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.164206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461d34-7179-4bfd-9a39-714218bce37a None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.166363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04461d34-7179-4bfd-9a39-714218bce37a None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:29.168981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04461d34-7179-4bfd-9a39-714218bce37a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:29.172414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 224/438] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:28 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:29.174405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.174883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.175360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.192264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.192906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.193184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.193459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.205254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:29.205489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:29.207417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:29.210192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:29.211929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:29.213275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 215/439] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:29.215309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.260649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:29.312886 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 225/440] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:29.313775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.356796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6867-21eb-4c26-8915-5b1d9472a070 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.403476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.404174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.404444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.404832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] PATH_INFO: `/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.406112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] RBAC: Authorizing `identity:get_project(project_id=fc7b19705b76444eba9e2a0476a6ff3e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:29.407612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:29.409589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 216/441] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/projects/fc7b19705b76444eba9e2a0476a6ff3e => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:29.410187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.448847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864050-8e89-4d2a-87f9-f4a1b6233201 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.480803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:29.495272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.496245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.496660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.497115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.498226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=39a47ee6b3f14e6ead39370f270c2f38, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:29.498751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:29.500601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:29.509159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 226/442] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=39a47ee6b3f14e6ead39370f270c2f38 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:29.514272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.547479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01e2c5-f31b-4362-a3f3-f50c0ffa9688 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.551529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:29.609765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.611192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.611469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.611959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.613162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:29.613692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:29.616450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:29.638679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 217/443] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:29.643695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.725249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c7228-383c-4771-af67-73bc17b08b88 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.781696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.783118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.783592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.784114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.786324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:29.789088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:29.792612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 227/444] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:29.793468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:29.817801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a5a163-23a6-4997-b9fd-80acca0db7b1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:29.874266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:29.875369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:29.875921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:29.876398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:29.897624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:29.943866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:29.979134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b254658-55e5-4593-905e-d4b862ec7f37 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.000253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.002287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.037904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.038582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.038858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.040513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.053504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:30.065913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:30.067945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:30.069590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 228/445] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:30.070754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.177743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 218/446] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:31:30.179229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.190855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6a2a92-b839-4385-8b48-c548fb782bbb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.209015 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:30.228042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.228666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.228936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.229278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.229942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:30.230245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.231704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.238908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 229/447] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=fc7b19705b76444eba9e2a0476a6ff3e&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:30.239770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.272939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:30.341726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8d5b5-a2b3-4a41-9507-f02975d55e72 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.400449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.401089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.401364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.401516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96206386-931d-4a28-b593-00059a8b0f77 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.401653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.402320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:30.402637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.404247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.415003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 219/448] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:30.421506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.455359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.456401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.456864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.457297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] PATH_INFO: `/v3/projects/e22dee573a0047c6958f6c34395e3b67` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.459194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] RBAC: Authorizing `identity:get_project(project_id=e22dee573a0047c6958f6c34395e3b67)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.461868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.465011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 230/449] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/projects/e22dee573a0047c6958f6c34395e3b67 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:30.471815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.550072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:30.559697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47aaaf-90b0-4302-af36-3e619017f0a1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.606607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:30.621735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.622658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.622943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.623255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.627271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f996f-84f1-4070-9d7d-245f703cb08f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.690776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.693250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.693250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.693250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] PATH_INFO: `/v3/groups/2865aac734d549e8886139f68a2004d9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.694796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] RBAC: Authorizing `identity:get_group(group_id=2865aac734d549e8886139f68a2004d9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.699841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.699841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 231/450] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/groups/2865aac734d549e8886139f68a2004d9 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:30.705521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.714350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.715870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.841852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cb51c-533b-4747-99c9-d08b96844a2d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:30.855060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 220/451] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:30.863923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:30.876979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:30.898422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:30.901389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:30.902470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:30.902946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:30.903384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:30.904880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] RBAC: Adding query filter params (group.id=2865aac734d549e8886139f68a2004d9, role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=e22dee573a0047c6958f6c34395e3b67) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:30.905426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:30.908064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:30.919145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 232/452] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/role_assignments?group.id=2865aac734d549e8886139f68a2004d9&scope.project.id=e22dee573a0047c6958f6c34395e3b67&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:30.993082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c9fc-dc20-4481-96cc-abbb61cf8f70 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:31.036847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:31.037506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:31.037782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:31.038056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] PATH_INFO: `/v3/projects/afd3325807f94fa3b6a7d418f71aaa51` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:31.039362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] RBAC: Authorizing `identity:get_project(project_id=afd3325807f94fa3b6a7d418f71aaa51)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:31.040917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:31.042681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 221/453] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:31:30 2022] GET /identity/v3/projects/afd3325807f94fa3b6a7d418f71aaa51 => generated 344 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:31.049570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:31.173684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468a6aa-c588-40bb-9c55-32544a3d75c0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:31.222287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a6ef1-f06a-4d2e-b6fb-4682e3361e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:31.222927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a6ef1-f06a-4d2e-b6fb-4682e3361e4d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:31.223210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a6ef1-f06a-4d2e-b6fb-4682e3361e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:31.223473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a6ef1-f06a-4d2e-b6fb-4682e3361e4d None admin] PATH_INFO: `/v3/groups/93d04f042327486d844c6d8f81c53a95` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:31.224706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a6ef1-f06a-4d2e-b6fb-4682e3361e4d None admin] RBAC: Authorizing `identity:get_group(group_id=93d04f042327486d844c6d8f81c53a95)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:31.226170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a6ef1-f06a-4d2e-b6fb-4682e3361e4d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:31.228218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 233/454] 10.222.0.52 () {56 vars in 1180 bytes} [Fri Oct 28 09:31:31 2022] GET /identity/v3/groups/93d04f042327486d844c6d8f81c53a95 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:31.234589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:31.346003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:31.353502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4092-032a-4210-9948-962cda6122ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:31.391862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:31.392471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:31.392744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:31.393013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:31.393753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] RBAC: Adding query filter params (group.id=93d04f042327486d844c6d8f81c53a95, role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=afd3325807f94fa3b6a7d418f71aaa51) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:31.394056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:31.395504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49fd90-f7a8-41dd-bdd2-d10d16e15d19 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:31.402006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 222/455] 10.222.0.52 () {56 vars in 1376 bytes} [Fri Oct 28 09:31:31 2022] GET /identity/v3/role_assignments?group.id=93d04f042327486d844c6d8f81c53a95&scope.project.id=afd3325807f94fa3b6a7d418f71aaa51&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:35.089807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae787a1-6edf-4116-9a96-58206f75c4e9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:35.090854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae787a1-6edf-4116-9a96-58206f75c4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:35.091341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae787a1-6edf-4116-9a96-58206f75c4e9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:35.093348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 234/456] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:35.111704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a1657-898e-4af0-be4b-f5823ed18258 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:35.112795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a1657-898e-4af0-be4b-f5823ed18258 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:35.113326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a1657-898e-4af0-be4b-f5823ed18258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:35.146315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9d5a1657-898e-4af0-be4b-f5823ed18258 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:35.147979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9d5a1657-898e-4af0-be4b-f5823ed18258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:35.266204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 223/457] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:35.770083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae787a1-6edf-4116-9a96-58206f75c4e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:35.825735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe46f-3b99-458e-b1a2-fac8ea148854 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:35.826195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe46f-3b99-458e-b1a2-fac8ea148854 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:35.826345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fe46f-3b99-458e-b1a2-fac8ea148854 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:35.828065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 224/458] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:35.835202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b8810-7a20-497a-8761-0128dbb5d7ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:35.835501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b8810-7a20-497a-8761-0128dbb5d7ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:35.835852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b8810-7a20-497a-8761-0128dbb5d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:35.872915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b27b8810-7a20-497a-8761-0128dbb5d7ed None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:35.874749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b27b8810-7a20-497a-8761-0128dbb5d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:35.877792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:35.967654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae787a1-6edf-4116-9a96-58206f75c4e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:35.994863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 225/459] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:35.996182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95536-abec-4d0c-8438-1791ef20dbe9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:35.996457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95536-abec-4d0c-8438-1791ef20dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:35.996733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95536-abec-4d0c-8438-1791ef20dbe9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:35.998891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 226/460] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:35.999965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e04d-0050-4fab-86d0-5b7ec36db15c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.000243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e04d-0050-4fab-86d0-5b7ec36db15c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.000549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32e04d-0050-4fab-86d0-5b7ec36db15c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.006968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 227/461] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:36.008164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a3837-8420-43bd-845a-062546073544 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.008456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a3837-8420-43bd-845a-062546073544 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.008693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a3837-8420-43bd-845a-062546073544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.028669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a8a3837-8420-43bd-845a-062546073544 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:36.029823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a8a3837-8420-43bd-845a-062546073544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:36.031807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:36.034969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.035834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.036493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.066838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:36.069274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:36.072165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e48d2d66-56b7-4e8e-babc-e43c4c993b5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:36.091677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:36.093164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:36.094066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:36.094066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:36.094066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:36.102280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 235/462] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:36.102280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.103098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.104120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.128701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:36.131067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:36.171515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 228/463] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:36.173154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a3837-8420-43bd-845a-062546073544 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:36.218191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 236/464] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:36.294776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:36.334353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:36.418335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca69777-6f53-4203-bb21-24c8209f5948 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:36.462872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:36.463948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.464576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.465102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.481804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:36.485740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:36.516216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:36.518002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:36.519703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:36.519703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:36.519703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:36.523590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 237/465] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:36.535143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:36.565522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a3837-8420-43bd-845a-062546073544 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:36.624409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:36.625597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.626065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.626504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.627635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:36.628020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:36.630385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0c59f-ec30-4fee-be77-016da09c11e4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:36.667317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 229/466] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:36.667317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7aa1da-d589-48ee-964a-d0abb4bf74b0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.667317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7aa1da-d589-48ee-964a-d0abb4bf74b0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.667317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7aa1da-d589-48ee-964a-d0abb4bf74b0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.667317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 230/467] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:36.667317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7aa1da-d589-48ee-964a-d0abb4bf74b0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:36.689731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:36.697510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:36.716391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd6b9-b860-4dfa-bee0-e0e093362293 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:36.761730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:36.762845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.763520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.764687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.765814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:36.766174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:36.768656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:36.794369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 238/468] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:36.794369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:36.835379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7aa1da-d589-48ee-964a-d0abb4bf74b0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:36.878881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:36.879755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.880177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.880546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.894070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:36.895583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8beacade-1345-4e04-8b3a-dc1ff4875e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:36.913548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:36.915392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:36.917453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:36.917453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:36.917453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:36.920190 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 231/469] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:36.921469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.921753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.922031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.924461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf98e0-cda7-46c8-908d-fa7852536af7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:36.944701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:36.945935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:36.971103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:36.971913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:36.972357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:36.972819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:36.973905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:36.976368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:37.001420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:37.016033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 239/470] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:31:36 2022] POST /identity/v3/users => generated 246 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:37.017180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.051546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:37.078379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 232/471] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:37.079925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.216346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6da627-e768-4a83-8a4c-ce2a3aa0503a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:37.277264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7eaa9a-7878-4538-9f6a-c4c139835461 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:37.287108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:37.288270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:37.288801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:37.289338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:37.312763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:37.315457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:37.336435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:37.337406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:37.337878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:37.338325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:37.339425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:37.343299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:37.347446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:37.349111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:37.350739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:37.350739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:37.350739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:37.350739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 240/472] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:37.350739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.387529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 233/473] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:31:37 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:37.389525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.473430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:37.485532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:37.577540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba321d-7fdc-4289-94f4-04547bb9d87c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:37.587631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53374d-7a4b-49ca-8f03-a00b404a59be None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:37.653214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:37.654441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:37.654976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:37.655550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:37.656892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:37.657417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:37.659264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:37.660625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:37.662281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:37.663078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:37.664772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:37.666355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:37.667185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:37.670659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:37.684480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 241/474] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:37.686770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.700768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 234/475] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:37.708699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.869145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31957188-28d5-4a97-b701-515c55fd0215 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:37.881022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:37.894004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d768be-8b49-4c26-adb6-f97daef7928c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:37.914025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:37.914645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:37.914915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:37.915177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:37.915839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:37.917391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:37.924796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:37.948126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 235/476] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:31:37 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:37.956493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:37.981172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:37.982112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:37.983052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:37.983460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:37.985571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:37.991034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:38.031929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.sql.core [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 28 09:31:38.031929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 28 09:31:38.031929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [parameters: {'id': 'f777de19592046aa9aafdaa614d08178', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 28 09:31:38.031929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81610) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application conn.query(q) Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result.read() Oct 28 09:31:38.082155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return ref.to_dict() Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 09:31:38.083170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application next(self.gen) Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application yield resource Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application next(self.gen) Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise self.value Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application session.commit() Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 09:31:38.084145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.session.flush() Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application compat.raise_( Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 09:31:38.085046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 09:31:38.086220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application conn.query(q) Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result.read() Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 09:31:38.087085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [parameters: {'id': 'f777de19592046aa9aafdaa614d08178', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:38.093797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 28 09:31:38.095806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:38.097877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 242/477] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Oct 28 09:31:37 2022] POST /identity/v3/roles => generated 144 bytes in 400 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:38.101926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:38.289294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0a2a9-a12b-4461-b82f-123396d6389a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:38.361831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:38.362897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:38.368356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:38.368356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:38.368356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:38.369536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:38.381138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782646c-7a13-4183-a664-0e64d2f256f5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:38.416023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 236/478] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:31:37 2022] POST /identity/v3/users => generated 248 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:38.464335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b47dc-40ac-4b2a-9eab-2ba007fcc571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:38.465820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b47dc-40ac-4b2a-9eab-2ba007fcc571 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:38.466553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b47dc-40ac-4b2a-9eab-2ba007fcc571 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:38.467292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b47dc-40ac-4b2a-9eab-2ba007fcc571 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:38.491883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b47dc-40ac-4b2a-9eab-2ba007fcc571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cb8b47dc-40ac-4b2a-9eab-2ba007fcc571 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:38.503285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:38.505366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 28 09:31:38.507207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:38.515792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 243/479] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:31:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:38.520466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:38.596419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:38.763175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2c14b-3469-49d4-b882-03b0694dbbc8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:38.831380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:38.832485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:38.832955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:38.833403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:38.834603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:38.835132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:38.838130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00b812-abfe-4964-b2b2-680d690bbcd1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:38.854843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 237/480] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:39.979367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afac9e0-14c0-4d38-a2a8-ceba08c9dac1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:39.979958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afac9e0-14c0-4d38-a2a8-ceba08c9dac1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:39.979958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afac9e0-14c0-4d38-a2a8-ceba08c9dac1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:39.981463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 244/481] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:39.991059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d266cd-1a8e-49f5-80fb-9fd46465a229 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:39.991059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d266cd-1a8e-49f5-80fb-9fd46465a229 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:39.991059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d266cd-1a8e-49f5-80fb-9fd46465a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:40.010619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-97d266cd-1a8e-49f5-80fb-9fd46465a229 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:40.013048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-97d266cd-1a8e-49f5-80fb-9fd46465a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:40.149987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 238/482] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:40.488811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7f86-3bd6-40c2-a960-c736a817a6c4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:40.489200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7f86-3bd6-40c2-a960-c736a817a6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:40.489434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c7f86-3bd6-40c2-a960-c736a817a6c4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:40.491180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 245/483] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:40.496735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:40.497053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:40.497285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:40.530329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:40.531566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:40.540559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:40.612137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 239/484] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:31:40.695666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7f86-3bd6-40c2-a960-c736a817a6c4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:40.742717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:40.844114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c7f86-3bd6-40c2-a960-c736a817a6c4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:40.901189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:40.902670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:40.902947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:40.903221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:40.916465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:40.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:40.924345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:40.926178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 246/485] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:40.933884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:41.092146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68397261-eaf7-47fd-aa5f-5bfe8283c225 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:41.143203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:41.179469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:41.180125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:41.180435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:41.180680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:41.181343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:41.181639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:41.183081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:41.199813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 240/486] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:41.219899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:41.225037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:41.351105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0555ad89-4c4d-40e4-82dc-186c43435ce5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:41.395644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:41.396602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:41.397233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:41.397796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:41.412049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:41.414338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:41.431555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:41.432744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:41.433593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:41.433593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:41.433593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:41.436605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 241/487] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:41.445578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:41.451567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:41.465875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d98af-e664-48df-891c-cff1b8d30468 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:41.540770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:41.541426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:41.541700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:41.541970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:41.566708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:41.569178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:41.573603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:41.576581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:41.577365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:41.579894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 247/488] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:41.583031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:41.701770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed9df2-4491-4341-b7c4-d0d8bf2ebe4a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:41.823965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:41.825361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:41.826587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:41.827264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:41.828640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:41.829384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:41.832226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:41.837018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4965bf2-c5ae-4d8d-b812-fdc31c2c6232 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:41.856734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 242/489] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:41.863197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:41.947697 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:41.948952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:41.949538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:41.950202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:41.952252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:41.952919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:41.955695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0747c-0cac-4018-92cc-3152ce6a18e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:41.973102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 248/490] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:41.975373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cb6e6-9fbe-4307-ada6-8b8eab07797d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:41.975933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cb6e6-9fbe-4307-ada6-8b8eab07797d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:41.976329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cb6e6-9fbe-4307-ada6-8b8eab07797d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:41.983277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 249/491] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:41.983277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cb6e6-9fbe-4307-ada6-8b8eab07797d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.021014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:42.041606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e4e3b-7ce8-4db4-83a8-c14aa982bc94 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:42.061746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:42.083886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:42.085113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.086282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.086725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.098786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:42.101454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5a1568d1-be85-4c52-8be7-39fe07cac81c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:42.128806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:42.130044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:31:42.131265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:42.135252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 243/492] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:31:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:42.136660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.137234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.137767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.167495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:42.168896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:42.177854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cb6e6-9fbe-4307-ada6-8b8eab07797d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:42.230230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:42.231609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.232655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.233107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.247648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:42.249565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:42.256844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 244/493] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:42.256844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-62ec898e-e83d-4f79-ba80-fcff3f9e4a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:42.273943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:42.274917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:42.275805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:42.275805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:42.275805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:42.275805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 250/494] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:42.276680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb456d2c-0898-4d01-a0d7-0544c149af7c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.276863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb456d2c-0898-4d01-a0d7-0544c149af7c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.279803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb456d2c-0898-4d01-a0d7-0544c149af7c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.279803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 251/495] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:42.282026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb456d2c-0898-4d01-a0d7-0544c149af7c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.376714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:42.381569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33969d-2960-4ed3-af57-54a1229aef9b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:42.391086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:42.421940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb456d2c-0898-4d01-a0d7-0544c149af7c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:42.435661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:42.436427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.436895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.437260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.438291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:42.438652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:42.440221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb4593-8bc1-4c5a-a37c-b5fb68580bd4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:42.470660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:42.471293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.471565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.471875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.472536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:42.472835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:42.474259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c1f7b9-9bdc-4522-a29d-187435e89e3f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:42.484665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 245/496] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:31:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:42.489334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 252/497] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:42.490479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.490772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.491048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.495149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975b7c1-9bdd-456f-b8b4-32c99cb74fc4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.495523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975b7c1-9bdd-456f-b8b4-32c99cb74fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.495759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975b7c1-9bdd-456f-b8b4-32c99cb74fc4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.500066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 253/498] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:42.500066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975b7c1-9bdd-456f-b8b4-32c99cb74fc4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.512873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:42.514153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:42.592938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 246/499] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:42.594182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.637141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975b7c1-9bdd-456f-b8b4-32c99cb74fc4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:42.686270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:42.696234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:42.697015 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.697427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.697835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.722069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:42.730500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3578e-b247-46af-bbaf-09b6efd41619 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:42.771098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:42.771618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.771908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.772185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.781367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:42.783667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:42.795658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:42.797220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5984951c-b8ea-40df-9f34-ff8a423e30fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:42.799555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:42.800567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 09:31:42.801478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:42.808996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 247/500] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:42.810732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:42.811212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:42.811479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:42.835292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:42.837480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:42.913505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 248/501] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:42.914391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.928363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 254/502] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:42.928363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:42.961077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.028670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.051794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42669b-300b-4b2e-8ffc-750ba490e5c3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.059861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64538c-f1e1-46d1-bf78-95b3c73cbb58 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.098836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.099822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.100355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.100810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.103835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.104786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.105239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.105689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.106926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:43.107427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.110012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.118593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:43.128324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:43.129249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:43.130140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:43.130754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 249/503] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:43.134270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.180444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 255/504] 10.222.0.52 () {56 vars in 1139 bytes} [Fri Oct 28 09:31:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:43.180444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.271456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae0899-6b7d-4b20-902b-f2030b8c8a5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.295351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.310924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.311532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b482427-136f-498a-8ae5-f61c6f43c6c1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.311633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.311858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.312128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.323049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.324599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.327669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:43.341281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:43.343141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:43.344888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:43.344888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:43.344888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:43.344888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 250/505] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:43.344888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.356636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.358149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.358595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.359022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.375476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.377928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:43.401662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:43.402791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:43.403647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:43.403647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:43.403647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:43.404014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 256/506] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:43.405389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.471691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e29cb-884f-47cf-9570-8916a6b73f78 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.509766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.510380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.510659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.510931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.511612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:43.511952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.513375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.517906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8864d-ea41-4f6c-a375-eaf94e3fe885 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.521956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 251/507] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:43.522811 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.556245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.556883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.557221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.557552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.571488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.592941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.642001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10818a-98fe-44ab-a867-2006f0b3f627 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.642294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.644863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.685960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.686592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.686868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.687138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.698203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:43.704180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:43.705303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 252/508] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:43.706250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.768315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 257/509] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:43.769186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.816133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.822963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788c2ce-388a-4838-9329-95abfe7b7f78 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.842777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:43.862356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.862976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.863254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.863525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.864217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:43.864531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.865967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.878698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 253/510] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:43.879305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:43.882173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569e440-eabf-4446-bcdf-50204f43f896 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:43.921493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:43.922106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:43.922385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:43.922660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:43.923306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:43.923603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:43.925040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:43.937597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 258/511] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:43.938460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.002430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ebcad-b47b-4dd4-b3e0-dfad419d5eba None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.037922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.038557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.038829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.039108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.041628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:44.046365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d9092-d1e7-448a-903f-9a98b083f5ee None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.051251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.052931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:44.069202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:44.070444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:44.071339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:44.071339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:44.071339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:44.071339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 254/512] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:44.071339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.088670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.089466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.089846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.090203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.090985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:44.091432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.093701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.104326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 259/513] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:44.105183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:44.108032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.194881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7ca63-be17-449b-b122-a9c0d5246680 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.228792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c718d5-8e26-41e8-baf0-24adf7ee2bf7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.234004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.234643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.234922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.235193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.246539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.248680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:44.264834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:44.266482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:44.267354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:44.267354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:44.267354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:44.267354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 255/514] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:44.267354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.275088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.276019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.276339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.276657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.277733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.280242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.308188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 260/515] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:31:44 2022] POST /identity/v3/users => generated 247 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:31:44.309206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.338735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:44.416037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:44.447138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909df72a-9b89-43ba-b240-c598f3bebc74 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.476164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4ef5c-120c-4add-b90e-070270de44fa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.504489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.505497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.505957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.506411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.517928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.520424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:44.542534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:44.544313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:44.546623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:44.549475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.549475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.549475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.549475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.550364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 256/516] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:44.550572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:44.550572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.550991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.551214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.561751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 261/517] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:44.562629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.691088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:44.692302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8d2fa-95d8-47fa-bb64-1a16ffbb0878 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.719270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f335bb1-e270-4c75-a5b6-eabdd6e46b66 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.736985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.737889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.738314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.738757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.751715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.754336 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.757573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:44.765636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:44.766275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:44.766590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:44.766869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:44.768275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:44.768595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:44.770763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:44.778094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:44.779934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:44.783328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:44.783328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:44.783328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:44.783328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 257/518] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:44.783328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.792093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 262/519] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:44.793607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:44.954692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0e10f-8fb4-4598-8979-1aff95adcc8e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:44.967560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4d6fb-7e02-4ad0-88ff-ec0701d1ecd5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.012087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.012880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.013433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.013872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.015060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:45.015558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.018001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.024585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.025382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.025824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.026264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.040045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.042621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.046026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:45.054783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 258/520] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:45.054783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:45.069081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:45.070071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 28 09:31:45.070985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:45.077794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 263/521] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:45.079169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.159067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:45.221404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de664660-080e-4912-b1c0-f5a8ce33cab0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.244185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266384df-3efb-4d51-8262-0dd990fc53f3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.284552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.285499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.285946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.286522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.287772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:45.288277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.290686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.299922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.301322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.302040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.302719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.316450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.320611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.331295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 259/522] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:45.331295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:45.350166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:45.351088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:45.351971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:45.361821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 264/523] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:45.361821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.475227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:45.479513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab033d52-8dd7-45c3-bfc9-887d4400f46a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.502294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:45.531270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.531922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.532201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.532474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.533882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=9dc2bb9976f949548bc4743eb297b082) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:45.534483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.536283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.541594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea5bc9-f35c-4143-9c32-d0c9ea96bb0d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.544261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 260/524] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/role_assignments?user.id=9dc2bb9976f949548bc4743eb297b082&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:45.546519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.595883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.597146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.597608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.599016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.600084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:45.600644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.602352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.638909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 265/525] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:45.639845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.791901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c557d3-af39-4436-a727-8743ff21260d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.847997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.848560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.848828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.849103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.859942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.861984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.866270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12138593-3cc7-4db4-abc6-922d1a53ee34 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:45.880550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:45.882292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:31:45.884215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:45.888820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 261/526] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:45.890787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.898465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:45.922552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:45.923178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:45.923454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:45.923725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:45.924562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:45.924846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:45.926404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:45.963904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 266/527] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:45.966239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:45.988195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:46.040049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32101b72-3e66-4c38-8b84-799b8ea5c497 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:46.083706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:46.084334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.084613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.084888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.160403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b195-5f18-4625-b9a2-96955e2b0670 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:46.195672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:46.197432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:46.224422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:46.225482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.226034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.226591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.228003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:46.228624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:46.231306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:46.245164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:46.271876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 267/528] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:46.273422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:46.324069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 262/529] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:46.325248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:46.335069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:46.441364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855d24b-d26a-45ae-930f-87cfecb7ba5f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:46.465058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902dc8-3d8d-4810-9c65-0d0cf34c9e6c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:46.486020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993287fc-c68d-43c4-9b81-fd4bede30452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:46.488124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993287fc-c68d-43c4-9b81-fd4bede30452 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.494997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993287fc-c68d-43c4-9b81-fd4bede30452 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.495281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993287fc-c68d-43c4-9b81-fd4bede30452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.514894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:46.532030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:46.533109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.533950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.534476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.536500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=b07b6ea0a9a7475fbcc8d6baf537cc4e) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:46.537079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:46.539827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:46.554116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 268/530] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:46 2022] GET /identity/v3/role_assignments?user.id=b07b6ea0a9a7475fbcc8d6baf537cc4e&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:46.555296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:46.702199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993287fc-c68d-43c4-9b81-fd4bede30452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:46.703804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993287fc-c68d-43c4-9b81-fd4bede30452 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:46.772282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fa688-e0ac-438e-b10c-d65207d76a33 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:46.804361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:46.813989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:46.814770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.815124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.815401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.828057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:46.829553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-745980f2-72a9-422e-9ad8-512a0f93b410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:46.846572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:46.848762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 269/531] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704dbb1-3216-440f-8d7c-e6bf3a39076b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704dbb1-3216-440f-8d7c-e6bf3a39076b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.850816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b704dbb1-3216-440f-8d7c-e6bf3a39076b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.855782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 270/532] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:46.859595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704dbb1-3216-440f-8d7c-e6bf3a39076b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:46.904342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 263/533] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:46.906629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:46.907115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:46.907584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:46.937958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:46.939876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:46.996681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:47.006274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b704dbb1-3216-440f-8d7c-e6bf3a39076b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:47.050964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 264/534] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:47.052001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:47.096994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:47.097985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:47.098442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:47.098888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:47.100011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:47.100546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:47.103173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:47.159961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 271/535] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:47.168250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:47.182694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf8bf8-f5f2-458e-b4ba-38c9cd15dbff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:47.210637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:47.229027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:47.230383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:47.230872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:47.231321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:47.247999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:47.249910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:47.301679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:47.302659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:47.303511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:47.303511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:47.303511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:47.311139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 265/536] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:47.312853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:47.366318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:47.367461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4494c-754d-471a-b09f-565efb3bdada None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:47.417104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:47.420509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:47.421107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:47.421860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:47.440373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:47.442249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:47.477248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:47.478156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:47.479085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:47.482701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 272/537] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:47.493351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:47.599915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60286f5-572d-4306-8959-2a34cb02fcf2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:47.689784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:47.690906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:47.691377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:47.691894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:47.693072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:47.693497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:47.695580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:47.712145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:47.716919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 266/538] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:47.718449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:47.765736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a0a16-d864-4215-a98f-cf2d8eac59e6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:47.828848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:47.830206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:47.830658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:47.831099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:47.832329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:47.832828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:47.835439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:47.864265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 273/539] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:47.865777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:47.881656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:47.920979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff4b24-ad8b-438e-8834-afc344659dcb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:47.988559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:47.990048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:47.990800 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:47.991435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.011977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:48.021771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:48.023820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:48.025647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:48.035943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 267/540] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:48.038117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:48.068267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f5974-4b58-49d2-8c34-893074e96b49 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:48.115537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:48.124760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:48.126161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:48.126578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:48.127030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.138334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:48.140482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a1dbc18a-83a2-4794-82a8-7b5dea4d40e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:48.167510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:48.169783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 274/541] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f65569-5c13-4e00-9b18-c0c5dc3d66af None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:48.171388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f65569-5c13-4e00-9b18-c0c5dc3d66af None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:48.172527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f65569-5c13-4e00-9b18-c0c5dc3d66af None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.175291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 275/542] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:48 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:48.176332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f65569-5c13-4e00-9b18-c0c5dc3d66af None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:48.213860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5de5a-ef5a-4758-9809-8af2e3aad39b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:48.276141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:48.277182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:48.277653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:48.278118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.280535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=416d5cf4e87c4ce9928ca7b4f0ee8220) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:48.280738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:48.283420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:48.299786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 268/543] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:48 2022] GET /identity/v3/role_assignments?user.id=416d5cf4e87c4ce9928ca7b4f0ee8220&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:48.299786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:48.315489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:48.355915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f65569-5c13-4e00-9b18-c0c5dc3d66af None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:48.428936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:48.430230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:48.430784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:48.431326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.432676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:48.433294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:48.436452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304362a3-4e51-4bcf-acf9-38d132c17bed None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:48.453152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 276/544] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:48.455711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab97ceb-d125-4731-a812-286a64834bae None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:48.456295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab97ceb-d125-4731-a812-286a64834bae None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:48.456823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab97ceb-d125-4731-a812-286a64834bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.491277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dab97ceb-d125-4731-a812-286a64834bae None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:48.493723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dab97ceb-d125-4731-a812-286a64834bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:48.517948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d9c2-3d5b-46f8-9fed-d475e5bd9ad0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:48.588720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:48.590067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:48.590669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:48.591919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:48.593533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:48.594180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:48.597217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:48.602419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:48.622564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 269/545] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:48.624177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:48.648645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 277/546] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:48.650856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab97ceb-d125-4731-a812-286a64834bae None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:48.832463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:48.954885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c272-0fac-4499-9d3b-f9b52050065e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:48.972408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab97ceb-d125-4731-a812-286a64834bae None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:49.045802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:49.047315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.048105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.048876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.050889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=16332ba5fe89453292d21325b333407a) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:49.051696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:49.054751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:49.062393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:49.063797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.064269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.064962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.066954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 278/547] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:48 2022] GET /identity/v3/role_assignments?user.id=16332ba5fe89453292d21325b333407a&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:49.099870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:49.103121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:49.160058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:49.161891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:49.163223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:49.163223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:49.163223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:49.167233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 270/548] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 544 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:49.173495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:49.222951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:49.279630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:49.393063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb3409-af78-449f-a755-102ecff251e2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:49.475638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:49.481062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.482707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.484049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.488658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:49.489644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:49.492755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:49.529712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 279/549] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:49.539699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:49.548790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37eba83-471a-4e85-a698-c486cc1dc63a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:49.604138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:49.623614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4526c65-a943-4979-a947-dbda52ce93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:49.624454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4526c65-a943-4979-a947-dbda52ce93f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.624853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4526c65-a943-4979-a947-dbda52ce93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.625225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4526c65-a943-4979-a947-dbda52ce93f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.638871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4526c65-a943-4979-a947-dbda52ce93f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d4526c65-a943-4979-a947-dbda52ce93f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:49.644115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:49.646075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:49.647830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 271/550] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:49.650041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b5039-4db7-4dd5-8ce2-8d5c084f44b4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.650419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b5039-4db7-4dd5-8ce2-8d5c084f44b4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.650788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b5039-4db7-4dd5-8ce2-8d5c084f44b4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.652525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 272/551] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:49.657706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b5039-4db7-4dd5-8ce2-8d5c084f44b4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:49.739324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa8b7b-0ca3-46ae-8abb-592db602e345 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:49.748140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:49.784010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:49.784682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.784956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.785253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.794917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:49.796423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-06f56149-2559-4f6a-bab3-151558299557 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:49.826854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:49.829219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:31:49.831227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:49.841509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 280/552] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:31:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:49.844551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.844551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.844551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.862555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b5039-4db7-4dd5-8ce2-8d5c084f44b4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:49.888080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:49.892072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:49.904390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:49.905001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:49.905388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:49.905719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:49.906487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:49.906849 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:49.908549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:49.924698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 273/553] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:49.927159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:49.993613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 281/554] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:49.994511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:50.020330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:50.192285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0ce38-8dd9-4877-a238-b9bc2ea29d89 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:50.203042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:50.213871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02daf474-1d26-400f-93f8-aef1ea5e2703 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:50.230969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:50.232147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:50.232442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:50.232713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:50.233452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:50.233753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:50.235227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:50.256158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 274/555] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Oct 28 09:31:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:50.273611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:50.285448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:50.286960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:50.287763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:50.288530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:50.309658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:50.313478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:50.353766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:50.355525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:50.357879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:50.357879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:50.357879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:50.365889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 282/556] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:50.373861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:50.448347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:50.454680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08ff67-3ac0-4f7f-ba04-92455f0a58cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:50.531838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:50.533145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:50.533430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:50.533696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:50.581648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbe09b-f875-4d62-a1ff-f257c4638ef6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:50.615868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:50.617366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:50.656935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:50.658455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:50.659369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:50.659980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:50.661255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:50.662010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:50.664922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:50.695828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 283/557] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:50.695828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:50.717779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:50.729431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:50.737960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 275/558] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 28 09:31:50.739173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:50.858535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1731107-85b6-4276-94ef-aa556af04382 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:50.897253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:50.897914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:50.898183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:50.898455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:50.908255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:50.909061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f7932-fde0-43e7-a5b1-fed780eb9658 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:50.909780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:50.923355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:50.924465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:50.925437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 276/559] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:31:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:50.926057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:50.983018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:50.984424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:50.984913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:50.985407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.001775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:51.009951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:51.022983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:51.025190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:51.027128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:51.031659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 284/560] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:51.033708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:51.097327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514aab1-9b6f-477e-8171-0c61971ead56 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.151947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:51.168737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:51.169841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.170331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.170825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.185682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:51.187691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:51.208437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:51.218779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:51.220624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:51.220624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:51.220624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:51.220967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78b7a7-59ba-4274-a954-a1200ede3c96 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.221126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 277/561] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:51.221126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:51.281767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:51.282655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.283152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.283497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.284370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:51.284842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:51.287290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5db2ee-7b3c-4408-a453-13110fbaabda None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:51.319041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 285/562] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:31:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:51.321156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d5d7-3d96-41ae-8ae4-0871eed99a27 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.321629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d5d7-3d96-41ae-8ae4-0871eed99a27 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.322131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d5d7-3d96-41ae-8ae4-0871eed99a27 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.329855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 286/563] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:51.330938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d5d7-3d96-41ae-8ae4-0871eed99a27 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:51.387594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2f697-8e6b-436d-99fa-2e5b43761de1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.405010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:51.449753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:51.451023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.451379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.451718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.452831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:51.453325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:51.455764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:51.478133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 278/564] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:51.478133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:51.510305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d5d7-3d96-41ae-8ae4-0871eed99a27 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.556429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:51.571731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:51.573696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.573696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.573940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.575018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:51.576097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:51.578743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63c2b11-8449-4a3c-b208-3f01e54ca353 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:51.612120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 287/565] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:51.614214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.614705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.615190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.615589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265d97d-c647-4d5f-a34b-69be0a535f32 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.647943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:51.650301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:51.655691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:51.656729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.657195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.657636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:51.678828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:51.681385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:51.705488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:51.707375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:51.709254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:51.709254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:51.709254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:51.712698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 279/566] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:51.714104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:51.761074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:51.782534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 288/567] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:51.783161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:51.929018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee4547-17d5-486d-81d0-cadd4482e6a4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.958207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d889c2-5a85-438c-a91d-6526d095259e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:51.978835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:51.996521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:51.998130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:51.998555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:51.999001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.009074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.010057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.010535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.011002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.018350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.021054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.024135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.026972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:52.047290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:52.048529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:52.059411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:52.059411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:52.059411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0a0daf21-e87c-48f2-a1bb-3620657ca757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:52.059588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:52.061186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:52.063221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:52.064579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 280/568] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:52.064725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 289/569] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:52.064865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.066054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac60d80-c3ea-48fa-909c-3ae9065fd34d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.066054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac60d80-c3ea-48fa-909c-3ae9065fd34d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.066054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac60d80-c3ea-48fa-909c-3ae9065fd34d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.066054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 290/570] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:52.066054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac60d80-c3ea-48fa-909c-3ae9065fd34d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.190685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:52.226575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25485af-b4f0-440a-909d-ec12fd6e1933 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.240565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac60d80-c3ea-48fa-909c-3ae9065fd34d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.266475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.267100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.267377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.267647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.268335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:52.268637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.270040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.282826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.283433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 281/571] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:52.283651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.284047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.284340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.284645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.285362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:52.285765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.287466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efea5b3-134d-4fc3-8910-364b1ebb2334 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.302832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 291/572] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:52.304504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.304895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.305274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.330268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:52.331707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:52.367576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:52.420956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d5135-ff70-48bb-a614-87b0fd59f631 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.423100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 292/573] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:52.425432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.487733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.488851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.489319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.489771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.491022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:52.491560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.494348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.497442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:52.519013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 282/574] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:52.520880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.556634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df1c47-1a68-4fc3-94a1-6a10b8212c6d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.599653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.600350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.600625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.600894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.613816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:52.618817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:52.620688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 293/575] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:52.622427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.665885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:52.693211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7d1ee-733f-47b9-b67f-05fa0470b8c6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.752390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.752949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.753223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.753644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.761434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.763565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.772919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279006d-04aa-439a-997c-3b1c9be9fd43 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:52.775951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:52.785698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 283/576] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:52.787590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.823179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.824414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.824855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.825878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.838911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.841398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.849947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:52.861029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:52.862893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 294/577] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:52.865296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.910582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae46409-6595-4500-8fad-381e520953ea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:52.954809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:52.955774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:52.956137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:52.956447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:52.957442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=0cb99f8c316545e2ba685c87006cbfca) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:52.957933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:52.960434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:52.969302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 284/578] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/role_assignments?user.id=0cb99f8c316545e2ba685c87006cbfca&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:52.970242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:52.990672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:53.018515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcfaa58-5cf7-464b-9e3a-8c14db22f3e4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.068265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.069308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.069777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.070240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.071536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:53.072089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.074783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.096337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 295/579] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:53.096337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.154322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10281ed4-73b6-4f07-ab0f-335d8de30e4a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.186475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:53.215233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.216439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.216865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.217293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.228740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:53.234799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:53.235963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:53.237088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:53.239245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 285/580] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:53.240271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.261648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f28ae9-568f-4889-bed6-7d05cc79c827 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.303070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.303628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.303952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.304174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.304881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:53.305187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.306632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.332141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 296/581] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:53.333432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.345629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:53.368909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b5f71-7921-480d-8b96-ccb5369f3838 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.410086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.410724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.410977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.411283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.412033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:53.412331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.413786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.441744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 286/582] 10.222.0.52 () {56 vars in 1139 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:53.441744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.459601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e081-aa43-4493-b24e-508b3412065f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.492194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:53.504610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.505863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.506469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.506873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.520156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.522954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:53.548735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:53.549742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:53.550642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:53.550642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:53.550642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:53.556214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 297/583] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:53.557436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.594045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c7478-e146-43de-a3cf-91edb3e05afe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.633010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.633976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.634328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.634710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.635516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:53.635979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.637964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.648423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 287/584] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:53.648967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.668239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:53.733848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b9ad5-c951-4a7e-ac47-207f03b3b4e3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.781393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:53.787962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.788193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b9d53-0e85-40c4-99cd-9b584589635b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.788984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.789425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.789860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/9dc2bb9976f949548bc4743eb297b082/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.813303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=9dc2bb9976f949548bc4743eb297b082, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.815561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.839797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 298/585] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:31:53 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/9dc2bb9976f949548bc4743eb297b082/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:53.839797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.855859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:53.858380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:53.859013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:53.859597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:53.887586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:53.891307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:53.922842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:53.923957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:53.924879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:53.924879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:53.924879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:53.932749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 288/586] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:53.934547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:53.980870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca85dad-7f37-4f38-b9ad-8e707763878e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:53.996056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.030366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.031392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.031940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.032496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.033575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:54.034101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.035796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.051433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 299/587] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:54.052705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.155314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124cb61-b151-423d-8152-6bd406d265c2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.185789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46766445-e10c-4d87-92f3-b7f44b27da5c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.230197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.231232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.231825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.232313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.233206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:54.233723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.233827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.234899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.235386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.235901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.236263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.251942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 300/588] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:54.253238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.264486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.266372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:54.287105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:54.288175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:54.289051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:54.289051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:54.289051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:54.292143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 289/589] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:54.293316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.301791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.306806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.391979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8631ddd-d329-4c19-8bbf-301b642310bb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.433033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.434074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.435142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.435562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.445203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.447163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.455048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4875f-61da-4a01-a495-0b2840ed57a3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:54.463004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:54.464216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 28 09:31:54.465177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:54.467857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 301/590] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:54.469073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.503422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.504197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.504540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.504899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.512152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.514371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:54.526448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:54.527419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:54.528410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:54.532474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 290/591] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:54.533445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.579413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.591899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d90098-7fc6-4baa-bcd6-4d60ce5c491f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.612157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.634241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.635276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.635849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.636320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.637196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:54.637732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.639360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.653286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 302/592] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:54.654460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.666924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef063ba2-fb55-4762-a899-db08819597b9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.712086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.712805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.713173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.713449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.714381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:54.714690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.716707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.740149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 291/593] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:54.741408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.775822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d58e9-81b9-417c-9a58-0dc8a53db539 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.817738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.818760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.819240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.819714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.820706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:54.821227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.822928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.838303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 303/594] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:54.839547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.858316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.866907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0a5a6-7181-4060-a4e8-e225e37963a7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:54.892026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:54.910895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:54.911887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:54.912307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:54.912755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:54.921824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:54.923654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:54.938289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:54.939492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:31:54.940556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:54.947499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 292/595] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:54.948774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:54.979060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041d25c-0cd2-47ac-a8f6-4f3a411c8038 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.016083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.018480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.018819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.019163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.031503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.033081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:55.052336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:55.053364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:55.054538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:55.054538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:55.054538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:55.054538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 304/596] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:55.055007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:55.067909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef977f-8dde-4386-87c0-6bc13b192b22 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.105574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.107086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.107514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.107959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/b07b6ea0a9a7475fbcc8d6baf537cc4e/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.124099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:55.124628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=b07b6ea0a9a7475fbcc8d6baf537cc4e, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.126436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:55.151806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 293/597] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:31:54 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/b07b6ea0a9a7475fbcc8d6baf537cc4e/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:55.151806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:55.197324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b93be4-0725-4d2f-9774-5470a3c18aed None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.224433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:55.281463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.282005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.282279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.282547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.283294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:55.283595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.285079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:55.326558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 305/598] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Oct 28 09:31:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:55.335189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:55.381638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4fb24-6c96-43ff-9111-2add52dc72e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.445655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.446638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.447123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.447596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.448526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:55.449081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.451244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:55.468355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 294/599] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:55.471952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:55.572098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee8074-97bb-4e80-885a-f995989e6a42 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.582046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:55.602975 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180f082-7a79-42fe-b526-0d5f5a9cd062 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.614305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:55.657679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.658002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.658442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.658724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.658999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.660423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.661396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.662156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.679658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.692375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:55.710155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.713032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:55.734572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:55.736401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:55.738163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:55.744129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 295/600] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:55.752441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3047cfb6-2540-4150-9bca-ce21f038a9d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:55.770850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:55.772828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:55.774890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:55.774890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:55.774890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:55.778290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 306/601] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:31:55.787069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e2f83-8a02-4917-8706-475f766c2f32 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.787884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e2f83-8a02-4917-8706-475f766c2f32 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.788577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e2f83-8a02-4917-8706-475f766c2f32 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.795571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 307/602] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:55.797034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e2f83-8a02-4917-8706-475f766c2f32 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:55.912168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf05d4b-29ef-40d7-8879-70040ecc8fbb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.963922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e2f83-8a02-4917-8706-475f766c2f32 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:55.977551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:55.978706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:55.979161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:55.979637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:55.980973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:55.981576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:55.984606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9b4d7-7e6b-4c11-817d-ffadef90943e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:56.006087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:56.007073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.007548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.008059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.008936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 296/603] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:56.009225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:56.009721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:56.011135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.011561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:56.011667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.012097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.027243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 308/604] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:56.028488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:56.051080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:56.053306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:56.060447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:56.066991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:56.170826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c0aa3e-d5cb-4eea-911a-a94a1ee58e98 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:56.200123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 297/605] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:56.201483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:56.252718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c9504-5ec9-4fc9-b8c4-e2a38ef4a060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:56.253791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9504-5ec9-4fc9-b8c4-e2a38ef4a060 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.254216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9504-5ec9-4fc9-b8c4-e2a38ef4a060 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.254667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c9504-5ec9-4fc9-b8c4-e2a38ef4a060 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/416d5cf4e87c4ce9928ca7b4f0ee8220/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.301842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c9504-5ec9-4fc9-b8c4-e2a38ef4a060 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=416d5cf4e87c4ce9928ca7b4f0ee8220, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:56.303302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c9504-5ec9-4fc9-b8c4-e2a38ef4a060 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:56.332169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 309/606] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:31:56 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/416d5cf4e87c4ce9928ca7b4f0ee8220/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:56.333799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209092bb-cd4e-42e1-bb8c-d666e4d96b31 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.334141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209092bb-cd4e-42e1-bb8c-d666e4d96b31 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.334422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209092bb-cd4e-42e1-bb8c-d666e4d96b31 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.336408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 310/607] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:56.352630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.352959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.353180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.388970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:56.390207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:56.456783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8cb08-348e-487a-9edc-94ed81a13f94 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:56.516232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 311/608] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 28 09:31:56.517339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:56.536741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:56.560621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:56.561499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.561884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.562262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.578294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:56.579998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:56.586485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:56.599524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:56.600737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:56.601839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:56.607500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 298/609] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:56.619202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:56.729424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fd413-e8bf-4ae7-8594-224a5279f4d3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:56.770518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250e4fb-6874-44f5-a9b0-a058a61ff90b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:56.807292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:56.808597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.809460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.809896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.821844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:56.828121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:56.829238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:56.830084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:56.833358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 312/610] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:56.844953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:56.854094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:56.854827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:56.855174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:56.855517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:56.856866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:56.857242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:56.863788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:56.889310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 299/611] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:56.897389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:57.005569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:57.012756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:57.063092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4bb0c-4a39-47cf-83cb-a5997d9aac69 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:57.096349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7360db6-3e1b-454d-a616-6155fbd59d37 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:57.121519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:57.123055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:57.123511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:57.123994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:57.125258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:57.125776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:57.128506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:57.144841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 313/612] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:57.146128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:57.168659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:57.170217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:57.170670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:57.171130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/16332ba5fe89453292d21325b333407a/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:57.199526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=16332ba5fe89453292d21325b333407a, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:57.202206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:57.232007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 300/613] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:31:56 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/16332ba5fe89453292d21325b333407a/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:31:57.233570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:57.352430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b676a-97bc-4ad4-9e35-0a7740b2a5ae None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:57.424808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:57.426114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:57.428950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:57.429559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:57.439120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:57.472420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:57.486882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:57.488195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:57.489043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 314/614] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:57.494983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:57.543259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7515f-0cff-4424-8bc9-fb530850761c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:57.572303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:57.632348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:57.633103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:57.633390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:57.633671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:57.652528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:57.655138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:57.688344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:57.690189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:57.691410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:57.691410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:57.691410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:57.691410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 301/615] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:57.697318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:57.722861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d8465-fe42-4787-ab8f-fd55f50f28eb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:57.791085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:57.792581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:57.793303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:57.794686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:57.796135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:57.796922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:57.800046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:57.818667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 315/616] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:57.831039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:57.878889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbac94-ebf1-4df4-a8a6-714d4b421688 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:57.942038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:57.942995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:57.943463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:57.943979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:57.945283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:57.945715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:57.948156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:57.949820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:57.972427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 302/617] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:57.981656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:58.026348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:58.042481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6030c9f-3f45-43e3-a089-4728507384bf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:58.109283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:58.110839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.111586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.112383 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.135857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:58.139575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:58.178321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:58.180219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:58.181930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:58.181930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:58.181930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:58.183269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 316/618] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:58.201070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:58.234212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e43e7-debb-45ac-95ec-f5d462e8ec5f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:58.327854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:58.329702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.330319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.330910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.339099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ab1f5-1abd-4b8c-b10d-d0d263a82bf8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:58.349254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:58.351968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:58.357137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4ace2d31-b81d-4820-8f5e-7fbf4dc8d620 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:58.370246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:58.373733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:31:58.376660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:58.388988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 303/619] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:31:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:58.388988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07a181-149b-4bdd-949d-0d47527c3aab None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.388988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07a181-149b-4bdd-949d-0d47527c3aab None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.388988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07a181-149b-4bdd-949d-0d47527c3aab None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.388988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 304/620] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:58.388988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07a181-149b-4bdd-949d-0d47527c3aab None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:58.413127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84277daa-14da-4a82-b349-3092a932711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:58.414473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84277daa-14da-4a82-b349-3092a932711b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.415588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84277daa-14da-4a82-b349-3092a932711b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.416104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84277daa-14da-4a82-b349-3092a932711b None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.417787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84277daa-14da-4a82-b349-3092a932711b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:58.418321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84277daa-14da-4a82-b349-3092a932711b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:58.422022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84277daa-14da-4a82-b349-3092a932711b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:58.456321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 317/621] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:58.459194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.459836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.460268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.487897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:58.489351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:58.509940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:58.534309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07a181-149b-4bdd-949d-0d47527c3aab None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:58.574230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:58.575557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.575960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.576327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.577193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:58.577653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:58.579273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:58.606284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 305/622] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Oct 28 09:31:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:58.607437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:58.647028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 318/623] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:58.648220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:58.810504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f25815-a7e7-4a0d-893c-8fc1041ee8a1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:58.877228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:58.885029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effe8fd-38dd-46c8-ad1a-25bb4fb4ab89 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:58.888309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:58.889759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.890233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.890706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:58.907845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:58.911000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:58.916297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:31:58.938227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:58.941812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:31:58.950613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 306/624] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:31:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:58.952899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:58.998709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:58.999236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:58.999513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:58.999816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.031531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:59.034270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9f42634e-cccd-41a6-a155-f5404d363410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:31:59.068737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:59.072285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 319/625] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f08c9-5505-4bf6-a403-c835f9efeca9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f08c9-5505-4bf6-a403-c835f9efeca9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5f08c9-5505-4bf6-a403-c835f9efeca9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 320/626] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:31:59.075735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f08c9-5505-4bf6-a403-c835f9efeca9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:59.150043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb1245-4846-497f-b780-99bb45053a24 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:59.186158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:59.186782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.187046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.187313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.188074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:59.188395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:59.189870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625185cf-d6b5-4ec5-8348-108fc783dfb9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:59.213888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 307/627] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:31:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:31:59.213888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.213888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.213888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.237987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:31:59.239939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5f08c9-5505-4bf6-a403-c835f9efeca9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:59.240061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:31:59.279784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:59.280439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.280729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.280995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.281652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:59.281957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:59.283354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:59.303801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 321/628] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:59.303801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:59.319229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:59.330928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:59.355253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 308/629] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:31:59.357148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:59.437425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9004172-bb4b-407b-81a7-b1ae196834a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:59.481691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:59.482345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.482614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.482887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.496274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:31:59.503243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:59.505259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 322/630] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:59.506940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:59.592780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:59.613441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba397a1-bf48-4540-a2d9-bbab6a34e71e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:59.631658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af832-87f2-4da7-a36d-8e87301f414f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:59.670650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:59.671289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.671565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.671878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.678299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:59.678929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.679204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.679471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.682151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:59.685414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:31:59.707606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:31:59.708626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:31:59.709512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:31:59.709972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 323/631] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:31:59.711421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:59.748852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:59.763971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:59.765649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:59.886076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3519b-b0a8-4408-b01a-3afb72591658 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:31:59.891922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 309/632] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:31:59.891922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:31:59.924362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:31:59.935182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:31:59.936317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:31:59.936841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:31:59.937242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:31:59.938350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:31:59.938836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:31:59.941146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:31:59.960018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 324/633] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:31:59.960018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.017684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:00.033076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9071b-0066-4c63-b312-72e54e9e7375 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.078333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.079146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.079584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.080084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.095189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbcf50-ef44-45c7-b1a4-8b3571801683 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.095418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:00.101524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:00.102649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:00.103582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:00.107930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 310/634] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:00.109095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.157999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.158966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.159401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.159874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.160952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:00.161446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.163350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.183963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 325/635] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:31:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:00.186011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.258625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b2e39-1b57-4de6-9013-628addcca842 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.264050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:00.302065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.303010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.303299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.303568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.318173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.320438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.329652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c1091-8669-445b-9ebd-30cd61d61411 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.334604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:00.343932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:00.345576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:00.347327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:00.347327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:00.347327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:00.351008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 311/636] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:00.351941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.382756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.383884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.384436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.384840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.405107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.408027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:00.436877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:00.437981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:00.438868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:00.438868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:00.438868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:00.440250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 326/637] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:00.440920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.487932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474d211-15e7-4b25-980c-30903544b818 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.552073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.553111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.553594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.554067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.555293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:00.555864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.558371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.560348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb2907-50c7-4c00-8603-3601a08e38db None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.563016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:00.575833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 312/638] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:00.577569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.604026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.604726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.605006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.605279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/0cb99f8c316545e2ba685c87006cbfca/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.622596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=0cb99f8c316545e2ba685c87006cbfca, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.624074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.651773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 327/639] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:32:00 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/0cb99f8c316545e2ba685c87006cbfca/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:32:00.651773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.700160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:00.778662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7bd73-87d3-426e-8592-89cec641fb18 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.781079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab51e0-3496-482e-ae91-ba13d114925e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.827241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.827906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.828168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.828505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.829112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:00.829274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:00.829423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.829781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.830050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.830318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.830826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.830997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:00.831267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:00.833034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7ccbd-cf10-4591-a79e-26fd3660b5a2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:00.844050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 328/640] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:00.844914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.856612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 313/641] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:00.857485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76472d-8bfe-467d-bc7a-fa2d75aaf709 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:00.857774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76472d-8bfe-467d-bc7a-fa2d75aaf709 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:00.858056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76472d-8bfe-467d-bc7a-fa2d75aaf709 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:00.859909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 314/642] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:00.860795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76472d-8bfe-467d-bc7a-fa2d75aaf709 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:00.868140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:00.974108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35605c1-49a3-4eb3-9c3d-5db806cb05fd None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:00.990658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76472d-8bfe-467d-bc7a-fa2d75aaf709 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.013601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.014279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.015124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.015399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.017550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:01.029852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.031405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.031518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.032094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.032369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.032635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.040398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.041887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:01.049438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:01.050540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:01.051447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:01.051447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:01.051447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:01.051447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 329/643] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:01.052228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f4981203-c1cb-4b33-abd7-998673273017 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:01.057110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:01.058122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:01.059328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 315/644] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:32:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:01.060029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.060029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.060411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.080567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:01.081711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:01.133764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:01.161619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 316/645] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:01.163118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.175566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c322b6-5c20-4859-84ed-6c24f82d6c5a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.214111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.214848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.215183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.216180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.226410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.228177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:01.242848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:01.243872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:01.244770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:01.249256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 330/646] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:01.250519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.318226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c39b9-68e8-4a40-8c0b-039df09a7df4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.335414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:01.362665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.363687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.364138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.364571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.365372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:01.365825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.367480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.387082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 317/647] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:01.388969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.396048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc0729-7ad5-4988-9093-d125f8febfdc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.429354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:01.441737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.443734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.444163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.444537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.445420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:01.445829 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.447396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.482170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 331/648] 10.222.0.52 () {56 vars in 1139 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:01.482170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.544921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25d0c0-b45c-4df2-924d-5091020c8a66 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.598260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.599355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.599735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.600143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.601739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:01.602188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.603838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.604332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa9250-8231-4ac3-9a8f-eea50dced8d0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.619172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 318/649] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:01.620524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.655608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.656659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.657184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.657656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.658744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:01.666842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.668357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:01.682648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:01.683727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 332/650] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:01.690604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.770224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:01.775184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdbff1-05b4-4d4b-aa58-0a61064fa7e6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.829443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.830384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.830831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.831271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.861115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950dd39-4fc4-4512-9b6b-a73fedca8970 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:01.917713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:01.918693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:01.919149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:01.919600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:01.920873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=9dc2bb9976f949548bc4743eb297b082) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:01.921369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.923958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:01.933797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 333/651] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/role_assignments?user.id=9dc2bb9976f949548bc4743eb297b082&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:01.935592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:01.961974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:01.964812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:02.023521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:02.153772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4365d-bfdd-4a19-b874-9f0c497051bf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:02.163084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 319/652] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:02.165191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:02.171660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:02.224408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:02.225084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:02.225359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:02.225614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:02.240091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:02.245508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:02.247722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 334/653] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:32:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:02.249497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:02.344773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d7fb0a-fa2c-4ed1-800b-0305a84fa693 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:02.391247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec8-db80-42e2-959d-771949bd45be None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:02.410250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:02.411289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:02.411806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:02.412565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:02.414093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:02.414628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:02.417815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:02.419168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:02.443066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:02.443876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:02.444160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:02.444492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:02.458623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:02.460212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:02.464279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 320/654] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Oct 28 09:32:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:02.467120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:02.493282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:02.495124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:02.509113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:02.509113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:02.509113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:02.509113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 335/655] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:02.509113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:02.546764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:02.641683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f48522e-3b1a-4493-b7df-ca9e8d12f642 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:02.719968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:02.720906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:02.721399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:02.721879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:02.723241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:02.724229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:02.726868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9215843-836a-484d-9f0b-e5ba09af3c64 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:02.727060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:02.746633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 321/656] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:32:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:02.749217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:02.798382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:02.799424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:02.799933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:02.800425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:02.870834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:02.947180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:02.950788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:02.988803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253cc0d-62d7-441d-b4b8-6312a0b65d70 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:02.997008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:03.067878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:03.069116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.069648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.070152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.071353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:03.071967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:03.075507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:03.106663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 322/657] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:03.108345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:03.183734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 336/658] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:03.183734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:03.330465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82af774-ed80-453d-a4c4-60ca995bbbee None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:03.358897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:03.363189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb813f-7ba3-46b0-ae04-9d22e5139d10 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:03.389950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:03.406630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:03.407647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.408130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.408588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.408749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:03.409320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.409683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.409880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.418055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:03.420199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:03.428238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:03.430759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:03.432943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:03.434618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 337/659] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:03.436424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-26c5efc7-b9da-4e0a-b37e-1f93799018a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:03.458090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:03.459693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:03.461816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:03.461816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:03.461816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:03.464195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 323/660] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:03.467071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ff4b-322a-4a15-8b03-4be373962965 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.467559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ff4b-322a-4a15-8b03-4be373962965 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.468070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ff4b-322a-4a15-8b03-4be373962965 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.470658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 324/661] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:03.475249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386ff4b-322a-4a15-8b03-4be373962965 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:03.611822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386ff4b-322a-4a15-8b03-4be373962965 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:03.641081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd74e8e-73db-4d19-babc-f341ddf5730b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:03.654716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:03.655507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.655987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.656400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.657287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:03.657734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:03.659729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:03.679799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 325/662] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:03.679799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:03.711224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:03.716394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:03.717438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.717898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.718413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.738172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:03.740859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-18ab84e5-2f69-4614-85fa-e35e37efe39f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:03.767858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:03.769027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:03.769949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:03.769949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:03.769949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:03.769949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 338/663] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:03.772136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.772462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.772930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.803552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:03.805941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:03.823775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d98ab-bb4b-4e8c-8453-09ec922655aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:03.859010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:03.871084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:03.872645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:03.873076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:03.873495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:03.874310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:03.874782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:03.876602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:03.891485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 326/664] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:03.892374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:03.947790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 339/665] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:03.947790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.050760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:04.075537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffa0fa-277e-42a4-86fd-9a103441e067 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.118096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b809a-5183-4772-becb-ad4874e9d3da None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.127218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.127770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.128066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.128337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.129094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=b07b6ea0a9a7475fbcc8d6baf537cc4e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:04.129473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:04.130958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:04.140974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 327/666] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/role_assignments?user.id=b07b6ea0a9a7475fbcc8d6baf537cc4e&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:04.141870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.261907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.262948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.263372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.263825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.264936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:04.265411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:04.267768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:04.276966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227105f-d6f2-4e11-ae2f-5819bced3c35 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.315811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 340/667] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:04.324015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.324439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.325219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.325494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.327797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.334959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:04.337204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:04.352325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:04.355712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:04.357654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 328/668] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:04.369135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.404914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:04.415598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:04.593608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f21a967-2341-467e-b5ca-c5fa9bc48b6c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.596677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc1bc1-0d3b-406b-a30f-38d242195e59 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.644196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.644855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.645258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.645520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.653914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.655056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.655221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:04.655485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.655984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.656815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:04.657371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:04.657895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:04.660605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c589bc-065a-4f5c-9d80-c167b2d67c0d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:04.675221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:04.676465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:04.677361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:04.677797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 341/669] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:04.679236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.695313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 329/670] 10.222.0.52 () {56 vars in 1137 bytes} [Fri Oct 28 09:32:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:04.698409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bb022-56f0-4f26-8f92-04b221f7183e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.698988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bb022-56f0-4f26-8f92-04b221f7183e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.699465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bb022-56f0-4f26-8f92-04b221f7183e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.701990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 330/671] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:04.703039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bb022-56f0-4f26-8f92-04b221f7183e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.807562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:04.849593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:04.860709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bb022-56f0-4f26-8f92-04b221f7183e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.878773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4e68-d3fc-4ce6-97db-7be804455dd8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:04.921460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.922255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.922717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.923100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.937565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:04.940252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:04.941506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.942179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.942816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:04.943616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:04.945511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:04.947251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 342/672] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:32:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:04.948666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:04.949660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:04.950716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:04.953400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17847a86-1828-4148-964f-e86dd1f80c79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:04.972962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 331/673] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:04.975715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:04.976230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:04.976710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:05.007476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:05.009673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:05.113302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 332/674] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:05.114554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:05.127054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc56cf4-fb9d-4d41-9507-797ae1beb254 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:05.179485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:05.180136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:05.180412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:05.180684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:05.197563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:05.250761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:05.327891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:05.329799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb982e-5ca5-48aa-b509-ca62998c46e3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:05.332034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:05.402143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:05.403217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:05.403724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:05.404297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:05.405564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:05.406331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:05.408695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:05.431681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 333/675] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:32:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:05.431681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:05.558873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 343/676] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:05.560593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:05.617094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3b011-b2c8-47e7-8b6d-0bdcd881bd09 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:05.642482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:05.653841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:05.674599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:05.675559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:05.676018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:05.676523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:05.677797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=416d5cf4e87c4ce9928ca7b4f0ee8220) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:05.678142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:05.680461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:05.690372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 334/677] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:05 2022] GET /identity/v3/role_assignments?user.id=416d5cf4e87c4ce9928ca7b4f0ee8220&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:05.691768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:05.786264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1fb6-e434-40ef-9dc1-6dcbab20e645 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:05.853713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:05.854749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:05.855192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:05.855635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:05.876141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:05.878789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:05.906889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:05.908789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:05.910575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:05.910575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:05.910575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:05.914050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 344/678] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:05.915555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:05.930315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b34e35a-054e-4ab0-adcd-a26196cf79ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.000718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:06.002253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.003055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.003844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.025742 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:06.037246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:06.039384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:06.041332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:06.044947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 335/679] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:06.046690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:06.086835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:06.100089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc57a-6616-4941-9705-768fd35d32c1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.138535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:06.182756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:06.183831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.184847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.186248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.209166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:06.214212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:06.243548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:06.245562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:06.248235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:06.248235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:06.248235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:06.248235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 345/680] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:06.248965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:06.282165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58e47-478d-46d9-8d9f-cce060cdcdbd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.364159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:06.365602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.366388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.367112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.369362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:06.370183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:06.374183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:06.406214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 336/681] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:06.411421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:06.478744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738c7ee-ed09-4db3-b18a-1640add5a85f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.556986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:06.557999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.558464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.559018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.561204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:06.561826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:06.566023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:06.585794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 346/682] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:06.587317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:06.609437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:06.631998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d7d57-4031-4f2f-99a0-a17c7f83c138 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.633935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:06.695296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:06.696723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.697196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.697672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.698861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:06.699386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:06.702058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9837433-b189-4932-946e-b15f622919f0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:06.733563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 337/683] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:06.736758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f139-3f7a-473b-ad59-611e440fea18 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.737194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f139-3f7a-473b-ad59-611e440fea18 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.737626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6f139-3f7a-473b-ad59-611e440fea18 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.743224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 338/684] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:06.743224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6f139-3f7a-473b-ad59-611e440fea18 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:06.748079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd94a1a-d19f-4a5d-865a-8727c7037666 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.810116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:06.810938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:06.811337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:06.811768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:06.821344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:06.823139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:06.837572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:06.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 347/685] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:06.842513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:06.942990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6f139-3f7a-473b-ad59-611e440fea18 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:06.966933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:07.004321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:07.005282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.005716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.006151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.025043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:07.027243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:07.029010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f023d-38a4-41a4-9280-cff1fc6339b9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c8c93749-29e2-4abe-95bc-996100fdc8d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:07.052858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:07.054863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:07.057396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:07.057396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:07.057396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:07.057396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 339/686] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:07.057396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.057396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.058073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.084961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:07.086962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:07.096457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:07.097426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.097870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.098303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.104315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:07.110141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:07.111841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:07.125914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:07.127653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:07.129434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 348/687] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:07.131325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:07.215838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 340/688] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:07.215838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:07.323556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c73234-407b-471d-8e09-c65fdd0b7245 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:07.388293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:07.402527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:07.405100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.405803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.406330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.407989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:07.408703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:07.411539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:07.429752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b63bb3-cb23-4197-9caa-97d69a2cf9c7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:07.458948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 349/689] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Oct 28 09:32:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:07.458948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:07.506167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:07.507616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.508381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.509117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.510839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:07.511671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:07.515001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:07.538573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 341/690] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:07.540642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:07.547492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:07.673830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b1387b-ff4a-4339-a117-f84307c7c6cc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:07.734895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:07.735991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.736408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.736808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.737802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:07.738269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:07.740198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:07.744269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d387e-85d2-4707-b780-4cfc3fe1ce25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:07.763247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 350/691] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:07.763247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:07.802508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:07.803357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:07.803797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:07.804270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:07.850914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:07.895461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:07.897205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:07.900593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:07.965159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97fef95-1c6e-4ce4-9d3c-230c8ebad9a8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.027610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.028818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.029314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.029817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.044986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.050490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 342/692] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:08.050490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.050852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:08.074561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:08.076431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:08.077976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 351/693] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:32:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:08.079130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.189834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1e469-94d1-47f1-9d3e-6a5de411df65 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.194169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:08.233222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.233758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.234040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.234288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] PATH_INFO: `/v3/services/network` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.234891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.236329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:08.248825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:08.250711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:08.252438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 343/694] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:08.254067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.281904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66e198-7b7b-4631-91eb-645f77ae2e53 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.291589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:08.345520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.346548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.347056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.347428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.348837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=16332ba5fe89453292d21325b333407a) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:08.349356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.351965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.375783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 352/695] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/role_assignments?user.id=16332ba5fe89453292d21325b333407a&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:08.375783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.410388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4206d0-c059-4731-8b5e-76c411c71a83 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.458543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.459585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.459948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.460286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.477811 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.480483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:08.506635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:08.508778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:08.510505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:08.510505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:08.510505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:08.514920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 344/696] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:08.515396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7ad16-7999-4685-9ca4-838a1e3dce70 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.516195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.553098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:08.571386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.572094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.572375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.572653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.573414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:08.573719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.575266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8dcba-0cd3-40c2-87f9-8b4aa97cd96e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.598772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 353/697] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:08.600075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded74b9e-bd2e-44c8-9e0c-8d684cb18c0b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.600400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded74b9e-bd2e-44c8-9e0c-8d684cb18c0b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.600663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded74b9e-bd2e-44c8-9e0c-8d684cb18c0b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.602292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 354/698] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:08.608449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded74b9e-bd2e-44c8-9e0c-8d684cb18c0b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.644247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:08.682300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d4a99-75d2-40c8-b92b-0cb0cec3d0ed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.742856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.743490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.743790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.744078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.744735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:08.745035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.746565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.756023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 345/699] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:08.757048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:08.776440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded74b9e-bd2e-44c8-9e0c-8d684cb18c0b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.845053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.846041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.846516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.846987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.848264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:08.848803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:08.851457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09956d-8e00-44b7-b2ea-34494dedb404 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:08.866038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:08.879077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 355/700] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:08.880679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.880986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.881271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:08.898623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e77994-c5c4-474a-9883-0d34cc6bbde2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:08.901114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:08.902336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:08.940617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:08.941228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:08.941492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:08.941801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:09.040204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 356/701] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:09.040204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:09.047023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:09.050073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:09.073221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:09.208317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:09.247428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 346/702] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:09.249370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:09.260194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb2d25-f0d6-41ad-9fa0-7c97f14d732f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:09.326903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:09.328488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:09.329338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:09.330063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:09.331409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] RBAC: Adding query filter params (type=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:09.332218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:09.335175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:09.357162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 357/703] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:09.359355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:09.518207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511e320-9a3c-40f3-8880-e6e893126033 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:09.585372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:09.592434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:09.593102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:09.593381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:09.593653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:09.603019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:09.604649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:09.618205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:09.619848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 347/704] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:09.621521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:09.623668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113fb42-dd7a-4801-b764-5bd86362f388 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:09.678800 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:09.686391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:09.687603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:09.688108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:09.688769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:09.707441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:09.709916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:09.737906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:09.740017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:09.741815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:09.741815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:09.741815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:09.741815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 358/705] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:09.741815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:09.767502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21a4de-be1a-45da-9135-b408ea553fa2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:09.829407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:09.830947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:09.831676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:09.832336 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:09.851992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:09.854969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:09.883205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:09.885467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:09.887394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:09.887394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:09.887394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:09.890227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 348/706] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:09.891994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:09.931482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceae1be-b225-4856-a429-75d2920ed3c5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:09.994657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:09.995711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:09.996270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:09.996835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:09.998027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:09.998537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:10.001134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:10.004235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:10.025620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 359/707] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:10.028055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:10.074969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:10.086611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4375-4f7a-4aa3-b105-5b99f0feaa61 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:10.147719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:10.149334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:10.149741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:10.150137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:10.150929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:10.151359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:10.154172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:10.183791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 349/708] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:10.183791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:10.261029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605eeaa-851f-4f51-ae9f-7023d9808f6c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:10.338926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:10.340736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:10.341206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:10.341689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:10.342890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:10.343796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:10.346847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:10.372588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 360/709] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:10.374555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:10.378105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e789d-a4ef-4905-8087-f9aa78270ba4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:10.447807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:10.448467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:10.449081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:10.450247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:10.452054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=0cb99f8c316545e2ba685c87006cbfca) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:10.452816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:10.455961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:10.475794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 350/710] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:10 2022] GET /identity/v3/role_assignments?user.id=0cb99f8c316545e2ba685c87006cbfca&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:10.475794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:10.513466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:10.517637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:10.700299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c89d3c-258d-422c-8779-6bb4ce64bab1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:10.746034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bdbcc-a9b0-48a0-8a4d-39b568037029 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:10.816617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:10.817293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:10.817572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:10.817846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:10.819156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:10.820536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:10.821152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:10.821756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:10.823094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:10.826919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:10.837995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:10.840451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:10.868589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 351/711] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:32:10 2022] POST /identity/v3/users => generated 254 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:10.873310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:10.875180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:10.878832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:10.884147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 361/712] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 510 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:10.894786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:11.064644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:11.112724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcd3c7-96b7-4628-aaec-335a99cc5a65 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:11.196373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:11.198198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:11.199044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:11.199896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:11.201609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:11.202466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:11.205604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408b529-27c1-4b05-898e-8953b6a0b1ee None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:11.233423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 352/713] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:11.243253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:11.492232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8f685-743a-4d72-a41a-63a45f12b4e9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:11.521145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:11.569173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:11.570696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:11.571414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:11.572163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:11.573812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=9dc2bb9976f949548bc4743eb297b082) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:11.575198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:11.583777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c6c2-b2ce-4fb3-b28b-060fc00a17f1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:11.597568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 362/714] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:11 2022] GET /identity/v3/role_assignments?user.id=9dc2bb9976f949548bc4743eb297b082&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:11.902013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae76018-c7a6-494a-af7f-6326ff5188fd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:11.903552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae76018-c7a6-494a-af7f-6326ff5188fd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:11.904378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae76018-c7a6-494a-af7f-6326ff5188fd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:11.907877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 353/715] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:11.931177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122511a-a836-4e30-bfaa-42b2a6ac835b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:11.932152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122511a-a836-4e30-bfaa-42b2a6ac835b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:11.932978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122511a-a836-4e30-bfaa-42b2a6ac835b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:11.976539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f122511a-a836-4e30-bfaa-42b2a6ac835b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:11.979200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f122511a-a836-4e30-bfaa-42b2a6ac835b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:12.156043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 363/716] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:32:12.279963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3028d-7185-4682-b531-18bf8604b209 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:12.280914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3028d-7185-4682-b531-18bf8604b209 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:12.281704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3028d-7185-4682-b531-18bf8604b209 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:12.284840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 354/717] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:12.294157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:12.294945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:12.295104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:12.335911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:12.337876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:12.494141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 364/718] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:32:12.691970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3028d-7185-4682-b531-18bf8604b209 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:12.952929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3028d-7185-4682-b531-18bf8604b209 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:13.040760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:13.042076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:13.042763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:13.044054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:13.045457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:13.048210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:13.063511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:13.092237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 355/719] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:32:12 2022] POST /identity/v3/services => generated 240 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:13.161032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:13.281065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:13.396010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcae43d-9ab3-4d3d-9225-50080dcc2f4a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:13.463102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:13.463797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:13.464066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:13.464498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:13.465684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:13.467143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:13.481837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:13.482916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:13.483809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 365/720] 10.222.0.52 () {56 vars in 1129 bytes} [Fri Oct 28 09:32:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:13.489958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:13.657014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66684d01-b017-4d2a-8381-3dedd6c1aa78 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:13.713059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:13.713698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:13.713974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:13.714244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:13.715215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:13.715730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:13.717652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac745a-7ce8-4212-9f97-77b4cf3782e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:13.725699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 356/721] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Oct 28 09:32:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:13.731381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:13.848570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d859f-fc4f-4689-a6d5-266ad116f9a2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:13.852310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:13.899495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:13.900152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:13.900428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:13.900695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:13.901338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:13.901634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:13.903070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bea164-1487-4e64-bb8d-4fec16d0953f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:13.908588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 366/722] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Oct 28 09:32:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:14.027185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb0dfc-b12a-427e-b048-2424b14842dc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:14.027855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb0dfc-b12a-427e-b048-2424b14842dc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:14.027855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb0dfc-b12a-427e-b048-2424b14842dc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:14.029496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 357/723] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:14.049423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49d4d5-f096-4de1-bf18-23a09acd6a32 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:14.050006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49d4d5-f096-4de1-bf18-23a09acd6a32 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:14.050486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b49d4d5-f096-4de1-bf18-23a09acd6a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:14.081042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8b49d4d5-f096-4de1-bf18-23a09acd6a32 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:14.082589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8b49d4d5-f096-4de1-bf18-23a09acd6a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:14.197830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 367/724] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:14.485539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb0dfc-b12a-427e-b048-2424b14842dc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:14.611102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:14.714380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb0dfc-b12a-427e-b048-2424b14842dc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:14.793815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ebd6a-02b1-417d-b885-6d67db0d1c65 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:14.794831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ebd6a-02b1-417d-b885-6d67db0d1c65 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:14.795338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ebd6a-02b1-417d-b885-6d67db0d1c65 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:14.797499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 368/725] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:14.803644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:14.804289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:14.804785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:14.805006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:14.805621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:14.805922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:14.806408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:14.843770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:14.856167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:14.857878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:14.859148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 358/726] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:14.865200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:14.873096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:14.874724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:15.054071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af402d52-cc90-43e1-8d00-c29b4ceff19b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:15.096245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:15.096947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 369/727] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:32:15.117823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:15.119227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.119962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.120644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.122652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:15.123381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:15.126136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8568da-1a9e-4feb-b4de-83326543b018 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:15.143159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 359/728] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:15.156945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:15.191844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a979c2-f742-426f-8266-894de7b9db99 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.192639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a979c2-f742-426f-8266-894de7b9db99 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.193319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a979c2-f742-426f-8266-894de7b9db99 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.196494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 360/729] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:15.206307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.207038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:15.207290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.208022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.240157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:15.242461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:15.365478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 361/730] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:15 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 28 09:32:15.383805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98385f-b084-498e-9ef8-790d9ce8fa14 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:15.435465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:15.436674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.437151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.437612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.440852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:15.463998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:15.468007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:15.495309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:15.496489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:15.497401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:15.497401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:15.497401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:15.503518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 370/731] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:15.504491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:15.637225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dcd67-f034-4116-bcca-c8a40e6cd7b6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:15.650710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:15.664915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:15.696202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591eda82-9dc6-4551-b159-e0f31e3a8284 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:15.701079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:15.702108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.702529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.702941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.719462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:15.727422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:15.728470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:15.729316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 362/732] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:32:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:15.730510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:15.745809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:15.746493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.746774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.747606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.748632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:15.748939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:15.751182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:15.773224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 371/733] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:15.774118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:15.872841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ec1bb-d013-45c7-8a03-142b196584a4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:15.920226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:15.920803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:15.921079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:15.921347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:15.936939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:15.943066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:15.945379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 363/734] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:15.947469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:15.967041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828c48c-01ad-47bc-bdf0-040ae2dadb99 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:15.987121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:16.028140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.029185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.029463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.029725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.030691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:16.030977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.032446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.034651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:16.046084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 372/735] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:32:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:16.047188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:16.124736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dcbfa-6fcd-4e19-b9e1-b8d46ad8834d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.192871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be420a75-9805-41d7-aa1c-e30f2fee6dcc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.197844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.199374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.200118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.200819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.217924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.221328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.239589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.240419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.240807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.241192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.242001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:16.242423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.244019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:16.251253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:16.253435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:16.255275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:16.259833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 373/736] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:16.260759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:16.263362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 364/737] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:16.267190 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:16.342556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:16.419830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:16.431783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e7fc0-b025-4b49-8710-ea2dfd7179f0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.477067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.477641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.477906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.478185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.492882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.494394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:16.516885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:16.517901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:16.518784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:16.518784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:16.518784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:16.523252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 365/738] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:16.524158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:16.594964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab16cb1-475d-49b8-8d29-5b779da1fb1f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.640983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.641749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.642026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.642298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.655245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.657774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.660622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa86b3-f770-411b-ac01-7f8cf25d429d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a7131895-fdbf-47e8-b08d-32792c37b02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:16.677154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:16.678871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:16.680112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:16.680112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:16.680112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:16.680112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 374/739] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:16.680112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd437f08-1260-4403-97cd-7862932d715e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.680112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd437f08-1260-4403-97cd-7862932d715e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.680741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd437f08-1260-4403-97cd-7862932d715e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.687152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 375/740] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:16.687152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd437f08-1260-4403-97cd-7862932d715e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:16.726043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.726675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.726949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.727275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.727996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:16.728301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.729771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.754744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 366/741] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:16.754744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:16.754744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:16.820942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd437f08-1260-4403-97cd-7862932d715e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.837323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:16.877520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.878141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.878416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.878685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.879336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:16.879631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.881088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a89db4-9a83-4bf3-9764-dc5929e80c22 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.893760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f528db-9a3a-4ff4-8a16-a15cb78601c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:16.894322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 376/742] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:16.895587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.895902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.896168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.919686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:16.920912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:16.935965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:16.936538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:16.936821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:16.937074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:16.937723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:16.938021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:16.939416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:16.956249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 367/743] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:16.956249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.009876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 377/744] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:16 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:17.010639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.026541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.091698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f021c-79f8-4876-b6a9-c9ebd93df2e5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.147496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5eb86-f4f4-4f10-b37e-990b1bd8f5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:17.148178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5eb86-f4f4-4f10-b37e-990b1bd8f5ab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.148458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5eb86-f4f4-4f10-b37e-990b1bd8f5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.148731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5eb86-f4f4-4f10-b37e-990b1bd8f5ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.150836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.155999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ce235-e57c-43c7-b153-35e88b5a14b7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.196068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:17.197098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.197382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.197632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.218069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:17.220581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:17.241958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:17.244039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:32:17.246148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:17.251532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 378/745] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:17.252918 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.270002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5eb86-f4f4-4f10-b37e-990b1bd8f5ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:17.272360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5eb86-f4f4-4f10-b37e-990b1bd8f5ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:17.368879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.401496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fe4d5-2f71-4918-854d-6b798c95606b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.435513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 368/746] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:16 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:17.437832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d6f31-5c1c-4989-8f5e-07718ec8219a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.438311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d6f31-5c1c-4989-8f5e-07718ec8219a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.438770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d6f31-5c1c-4989-8f5e-07718ec8219a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.441194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 369/747] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:17.442492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d6f31-5c1c-4989-8f5e-07718ec8219a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.452162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:17.453215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.453688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.454157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.467296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:17.469858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:17.484314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:17.486204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 379/748] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:17.488339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.512442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.568718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d6f31-5c1c-4989-8f5e-07718ec8219a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.606848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:17.607471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.607728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.608019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.608750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:17.609036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:17.610467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5c71e-8053-4dd5-8504-16842374139f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.610644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:17.631597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 370/749] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:17.633037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.652425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:17.653217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.653593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.653964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] PATH_INFO: `/v3/services/network` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.654661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:17.656269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:17.659929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-667ea6c5-9fb5-410c-8f62-2b146d2b11ff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:17.668037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:17.669971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:17.671611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 380/750] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:17.674112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.674112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.674112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.701202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:17.703140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:17.756790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feeee24-4a57-432d-b151-539e543ac705 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.792321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 381/751] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:17 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 28 09:32:17.794201 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.802950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:17.803588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:17.803896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:17.804164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:17.815593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:17.817377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:17.826588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:17.836302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:17.838388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:17.841062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:17.841062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:17.841062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:17.846476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 371/752] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:17.847445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:17.936002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:17.962808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8d163-5d11-4d67-bee4-e2fb4ab7cd53 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:17.984801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9287-8800-4410-ad6a-6220f88ec79b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.018691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.019446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.019858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.020336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.021630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:18.022182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.024674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.041485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.042470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.042944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.043378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.055376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 382/753] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:18.056823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.068082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.070597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:18.096062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:18.097090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:18.098031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:18.098031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:18.098031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:18.098031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 372/754] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:18.098914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.186620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:18.230065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86da90-5bd0-41e9-ad99-81a932a4309b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.265596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049fba62-428b-46d1-8e2d-bd8fd9994f78 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.270935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.271982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.272460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.273064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.274205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:18.274739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.277750 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.290581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 383/755] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:18.290581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.296197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:18.308234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.309194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.309516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.309737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.310382 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:18.310679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.312093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.324324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 373/756] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:18.325174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.426620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece987cd-35e3-49e8-ad6f-9fbf5046dfa8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.459735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de54bb5-e7b0-4da7-9d06-827d63c8795a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.467260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.467987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.468335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.468664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.472685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:18.502330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.503218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.503581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.504068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.504870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:18.505268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.506832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.521774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 374/757] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:18.523091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.565014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.567454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.593368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:18.662274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1604cb-44e1-4e83-8656-8878751486f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.705794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 384/758] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 28 09:32:18.707189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.711682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.712490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.712920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.713359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.714149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.716343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.733038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 375/759] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:32:18 2022] POST /identity/v3/services => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:18.734414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:18.753694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:18.851184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea4041-1428-4101-9383-b3b332d973a3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.922297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:18.923258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:18.923621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:18.924113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:18.925211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] RBAC: Adding query filter params (type=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:18.925695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:18.928149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:18.930294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3fdb6a-5569-4dc7-9e6e-7374c47e64aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:18.935251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:18.937092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 385/760] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:18.938612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.000017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.000976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.001389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.001782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.013573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.015817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:19.032136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:19.034095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:19.036096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:19.037087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 376/761] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:19.038230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.109165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:19.119183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1db5e-4fdd-453c-9591-e321c2c5a86f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.174484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.175423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.175860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.176269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.185638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.187295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:19.202424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:19.204620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 386/762] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:19.206742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.221026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034e92e-f767-4600-9a40-d33c16d86fa2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.282319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.283270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.283785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.284251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.304157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.306915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:19.336875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:19.338813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:19.342224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:19.342224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:19.342224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:19.342767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06ea32-1738-4f91-a205-083b50431049 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.346232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 377/763] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:19.347626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.382054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:19.387997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.389583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.390003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.390394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.391228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] RBAC: Adding query filter params (interface=public, service_id=a729ea492e2d4ad59133de4f2b7e6c94, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:19.391876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.393397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.410100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 387/764] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/endpoints?service_id=a729ea492e2d4ad59133de4f2b7e6c94&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:19.410100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.446867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:19.502544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23e8844-6479-4ad0-863e-5ee1338416cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.548338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.549344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.549806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.550452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.551796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:19.552480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.555329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.578071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 378/765] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:19.579989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.593522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd16f70-1054-4f1d-b643-f9d313d3db23 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.651353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.652422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.652884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.653344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.654589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:19.655113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.657763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.688268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 388/766] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:19.689836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.693234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:19.725628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104ba12-177f-4942-a8b8-b892253193af None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.786201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.787722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.788501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.789226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.790788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:19.791409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.794577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.815188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 379/767] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:19.816853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.833739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:19.869026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387deb5-9256-4d5f-9a40-683d50b8987d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:19.936448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:19.937770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:19.938296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:19.938746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:19.939807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:19.942651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:19.963808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 389/768] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/services => generated 772 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:19.963808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:19.981102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144408c1-4ccd-4e2e-a01f-71e9393a617c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:20.046381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:20.047628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:20.048220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:20.048752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:20.050227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=0cb99f8c316545e2ba685c87006cbfca) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:20.050715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:20.054424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:20.063526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:20.072063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 380/769] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:19 2022] GET /identity/v3/role_assignments?user.id=0cb99f8c316545e2ba685c87006cbfca&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:20.074003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:20.285182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76548337-943b-4150-8012-e3da16f34f58 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:20.286353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400d533b-bffb-4837-9bc8-bf1aec45447a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:20.356129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:20.357101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:20.357560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:20.358051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/9dc2bb9976f949548bc4743eb297b082/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:20.361515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=9dc2bb9976f949548bc4743eb297b082, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:20.364571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:20.365459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:20.366823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:20.367552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:20.368281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:20.382129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:20.385248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:20.391376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 390/770] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:32:19 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/9dc2bb9976f949548bc4743eb297b082/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:20.408270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:20.409826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:20.411376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:20.415254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 381/771] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:20.424311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:20.429977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:20.634572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438e8af8-8713-42cd-9663-4ee622dd0ed0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:20.702173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:20.703171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:20.703640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:20.704151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:20.705374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:20.705893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:20.708602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b32fdc-8034-453d-b335-6402d4229d8b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:20.732207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 391/772] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:20.739581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:20.874452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:20.956408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e886c6-bff8-4b98-ac70-ce11a00e47fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:21.041005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:21.042034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:21.042507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:21.043536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:21.044997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=72d842d8875d4f3f8d44875c410ebf37) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:21.045519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:21.048294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05224048-05f3-4da3-80ae-c85014a568cb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:21.064161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 382/773] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:20 2022] GET /identity/v3/role_assignments?user.id=72d842d8875d4f3f8d44875c410ebf37&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:21.774524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2281c-2836-4d5c-87cb-b4af275a06e3 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:21.775324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2281c-2836-4d5c-87cb-b4af275a06e3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:21.775428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2281c-2836-4d5c-87cb-b4af275a06e3 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:21.778465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 392/774] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:21.785674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:21.786448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:21.786448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:21.812876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:21.814102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:21.903467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 383/775] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:22.481293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2281c-2836-4d5c-87cb-b4af275a06e3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:22.617504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:22.687878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2281c-2836-4d5c-87cb-b4af275a06e3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:22.753574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:22.754460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:22.754919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:22.755374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:22.757044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:22.759736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:22.778518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:22.781071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:22.781959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 393/776] 10.222.0.52 () {56 vars in 1129 bytes} [Fri Oct 28 09:32:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:22.787380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:22.893000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:22.978196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e947b-e9fa-499c-ba64-7c4f373f443a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:23.064844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:23.066353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:23.067083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:23.067820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:23.069211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:23.070032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:23.072940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:23.087292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 384/777] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Oct 28 09:32:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:23.094116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:23.365983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca861e3-1d06-478d-a0f8-1cba224e276b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:23.416903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:23.431623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:23.432720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:23.433736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:23.434832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:23.436063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:23.436608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:23.439520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:23.454895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 394/778] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Oct 28 09:32:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:23.462084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:23.866631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87633e-a9bb-4d6f-a723-cce556b2b48a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:23.905749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:23.944771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:23.945804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:23.946277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:23.946751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:23.948060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] RBAC: Adding query filter params (interface=public, service_id=e89e0653ff554c8eaef59a99db10e261, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:23.948601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:23.951178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83cb16-3502-4bc0-a73f-4916dbfaf58e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:23.966198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 385/779] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:32:23 2022] GET /identity/v3/endpoints?service_id=e89e0653ff554c8eaef59a99db10e261&interface=public®ion_id=RegionOne => generated 194 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:23.973545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:24.221766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db490c4-99f9-4985-86b0-0c98602b6482 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.222183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3401a-13a2-443f-9907-42d80ac905a1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:24.222295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db490c4-99f9-4985-86b0-0c98602b6482 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.222698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db490c4-99f9-4985-86b0-0c98602b6482 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:24.225821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 386/780] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:24.232879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f554b9c-2ea1-404f-adb2-c3340eee2962 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.233316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f554b9c-2ea1-404f-adb2-c3340eee2962 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.233744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f554b9c-2ea1-404f-adb2-c3340eee2962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:24.285957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6f554b9c-2ea1-404f-adb2-c3340eee2962 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:24.287955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6f554b9c-2ea1-404f-adb2-c3340eee2962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:24.295974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893389d6-3b9d-400b-aba8-37845a251ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:24.296936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893389d6-3b9d-400b-aba8-37845a251ded None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.297371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893389d6-3b9d-400b-aba8-37845a251ded None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.297795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893389d6-3b9d-400b-aba8-37845a251ded None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:24.298796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893389d6-3b9d-400b-aba8-37845a251ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:24.301201 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893389d6-3b9d-400b-aba8-37845a251ded None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:24.323636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 395/781] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:23 2022] GET /identity/v3/services => generated 772 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:24.426663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 387/782] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:24.744634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa0b0d-43f7-4ec0-816e-017091a104b9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.745033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa0b0d-43f7-4ec0-816e-017091a104b9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.745451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa0b0d-43f7-4ec0-816e-017091a104b9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:24.748241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 396/783] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:24.757165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541101e6-087d-4073-bbc9-fddc48781c59 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.757571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541101e6-087d-4073-bbc9-fddc48781c59 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.757969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541101e6-087d-4073-bbc9-fddc48781c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:24.792312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-541101e6-087d-4073-bbc9-fddc48781c59 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:24.794336 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-541101e6-087d-4073-bbc9-fddc48781c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:24.919907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa0b0d-43f7-4ec0-816e-017091a104b9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:24.935120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 388/784] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:24.950777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87456b-13ca-4bc1-a103-e70b6f7002a4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.951239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87456b-13ca-4bc1-a103-e70b6f7002a4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.951694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87456b-13ca-4bc1-a103-e70b6f7002a4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:24.955215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 389/785] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:24.966013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446aa4e-2476-4037-8e4e-afea27740134 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:24.966484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446aa4e-2476-4037-8e4e-afea27740134 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:24.967106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446aa4e-2476-4037-8e4e-afea27740134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.005012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5446aa4e-2476-4037-8e4e-afea27740134 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:25.006866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5446aa4e-2476-4037-8e4e-afea27740134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:25.032818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:25.091399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa0b0d-43f7-4ec0-816e-017091a104b9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:25.103340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:25.147584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:25.148529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:25.149013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:25.149530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.150327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:25.152048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0a7794ae-8ab6-4c43-92e2-8a0f3f0e54e9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:25.162679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:25.163799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 28 09:32:25.164699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:25.168002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 397/786] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:24 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:25.169935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2b0be-388b-487c-b4ac-7e957df3a2a1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:25.170634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2b0be-388b-487c-b4ac-7e957df3a2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:25.171156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2b0be-388b-487c-b4ac-7e957df3a2a1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.177527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 398/787] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:25.178160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 390/788] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:25.179309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2b0be-388b-487c-b4ac-7e957df3a2a1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:25.186165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:25.186627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:25.187096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.233582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:25.235334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:25.353690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 391/789] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:25.381322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2b0be-388b-487c-b4ac-7e957df3a2a1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:25.411954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:25.438797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:25.439842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:25.440427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:25.440920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.441820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:25.442320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:25.443989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:25.450356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 399/790] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:25.458638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:25.470631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:25.767404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f37f2d-5a30-4d22-889d-096b32e4f0f1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:25.773844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ecd5a6-3f12-42b3-a6e7-d4f96aade3f4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:25.776754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:25.844979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:25.845660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:25.845972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:25.846457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:25.846657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:25.846925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.847040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:25.847422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:25.848109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:25.848724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:25.851346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:25.868305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 392/791] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:25.868498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:25.869973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:25.878409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:25.879429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:25.880295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:25.881531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 400/792] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:32:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:25.882552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:25.962646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:26.033421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa09589-0929-40df-96e3-e02092efc227 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.075649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209207-e6b4-40f4-bc2a-d3300bda7d8b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.098033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.099080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.099560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.100260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.101728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:26.104432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:26.139571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 401/793] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:32:25 2022] POST /identity/v3/endpoints => generated 337 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:26.140968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.151373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.153443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.154170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.155094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.172448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:26.181920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:26.184078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:26.185844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 393/794] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:26.187340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.237171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:26.335104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:26.344078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfb3dc-ad33-40ce-8d60-851b52059976 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.378061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111263c4-210f-4e8d-91b7-f157e5502818 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.420393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.421517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.422086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.422693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.424507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:26.425274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:26.428156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:26.428958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.429678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.430024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.430361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.445884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 402/795] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:26.447512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.448499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:26.454460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:26.455474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:26.456342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:26.458694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 394/796] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:26.460341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.607350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:26.615425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272991c-78d0-41c3-83a3-75741695668d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.634122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28778433-894a-4cdb-81ba-5d8e0a1b9298 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.656792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.657421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.657693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.657962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.658632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:26.658961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:26.660379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:26.675719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 403/797] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:26.675719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.687976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.688977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.689369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.689772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.702626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:26.704455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:26.707333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:26.725978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:26.727840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:26.729641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:26.729641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:26.729641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:26.735074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 395/798] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:26.736429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.810250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5a839-f28a-402a-b3ff-a3496a986db1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.851007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.851625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.851932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.852236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.853117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:26.853436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:26.855328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:26.872591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 404/799] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:26.872591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:26.881216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05d68b-7497-441e-af61-e0f84a568e33 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:26.935761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:26.936464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:26.936741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:26.937008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:26.939334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:26.952900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:26.954432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:26.975218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:26.977350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:26.979055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:26.979055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:26.979055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:26.981906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 396/800] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:26.982888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.038916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:27.068205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233dfde-7a83-4c8c-9429-d7270edccbdb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.127533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.128071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.128351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.128626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.129256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:27.129572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.130944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.154256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 405/801] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:27.154256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.154548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8596f-5f55-48aa-8393-626d44feebd3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.216032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.217032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.217470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.217973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.235467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.237315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:27.261264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:27.262893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:27.264199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:27.264199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:27.264199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:27.270355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 397/802] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:27.271488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.336344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9746e0-1f7c-4cee-a398-3eb06f551691 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.343322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:27.394447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.395308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.395787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.396217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.397283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:27.397779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.401435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.425025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 406/803] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:27.426376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.447434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:27.450096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57824ec8-ddd1-4f59-a324-5cbedae05075 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.502567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.503402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.503875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.504282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.516142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.518638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:27.535234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:27.537283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:27.539131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:27.540249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 398/804] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:27.541342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.594629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d37fb-0e3e-43c8-96ab-6ad53713047d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.633513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.634134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.634411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.634685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.635338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:27.635639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.637109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.649381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 407/805] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:27.650145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.686733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:27.691461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9316d9c-3462-4cdb-ad0c-1e16a0c18bca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.729629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.730851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.731130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.731395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.734284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:27.741847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.743315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7cd3317f-4b9b-4010-8661-67ddb4a2b8d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:27.760527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:27.761826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:27.762881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 399/806] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:27.763514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783cef-8740-4edd-8587-77ea062e6750 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.763514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783cef-8740-4edd-8587-77ea062e6750 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.763514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f783cef-8740-4edd-8587-77ea062e6750 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.764904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 400/807] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:27.765611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783cef-8740-4edd-8587-77ea062e6750 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.782585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fa7ce-c5c0-433f-bded-4ecbe63b0d14 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.824189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.824810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.825083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.825348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.826088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:27.826386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.827914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.847694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 408/808] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:27.848565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:27.890293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f783cef-8740-4edd-8587-77ea062e6750 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:27.926363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:27.927040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.927295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.927567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.935688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:27.937200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b3513b15-21ed-4c31-9935-9914a45939e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:27.949866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:27.955787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 401/809] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:27.956949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:27.957779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:27.974170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:27.976229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:27.980815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3bb95-6fb4-429e-8e8f-0cec97195fd4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.001427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.006191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.018685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.019414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.019834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.020195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.021030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:28.021424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.022997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.042030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 409/810] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Oct 28 09:32:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:28.043587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.092363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 402/811] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:28.092363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.161698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960118f-2029-49b3-ade6-82b5359f1feb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.216384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.217346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.217842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.218275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.224431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cda0fc-5b03-411f-a791-f412de2f8575 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.236331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.239143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:28.266654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:28.267721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:28.268699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:28.268699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:28.268699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:28.270141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.270764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.271036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.271299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.272077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:28.272380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.273867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.276124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 410/812] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:28.277410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.301912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 403/813] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:28.301912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.315283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.330192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.409946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faeb21-4db2-4f22-80dd-e96827df63a2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.452654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.453460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.453872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.454263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.469044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.470731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.476779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea8429-3a8d-401f-9521-fe4a5a752f30 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:28.489348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:28.491369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:28.493745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:28.493745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:28.493745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:28.493745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 411/814] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:28.493745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.526176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.527392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.527665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.527985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.528611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:28.528903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.530279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.542117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 404/815] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:28.543144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.610633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.632999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.634307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b518eb-af23-4b68-8c46-31eab8c9485b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.674033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4107c-2ae5-46c7-bd85-d06d44dfc3c8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.686235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.686996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.687412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.687914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.711059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.711694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.711993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.712265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.712861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.714778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:28.723938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:28.725175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:28.726737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 405/816] 10.222.0.52 () {56 vars in 1129 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:28.729410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.772951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.774431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.867082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80241d6-9357-41ef-a4fd-98351867a0fb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:28.885259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.901029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:28.904529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 412/817] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:32:28.905497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:28.908468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:28.910158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:28.910436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:28.910703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:28.911382 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:28.911684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:28.913091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:28.930088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 406/818] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:28.931915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.023675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a74214-989c-40b5-83d2-4aece28644d0 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.065298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.066320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.066779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.067251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.075769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.077682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.081680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e60f36-8f3f-4e56-b5e3-0917c50846f7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:29.096880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:29.098072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:29.099107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 413/819] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:29.100053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.126840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.127447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.127720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.128016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.128649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:29.128947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.132469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.135818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 407/820] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Oct 28 09:32:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:29.136652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.180874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:29.198194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:29.243274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0e3f0-77e7-4578-82ae-0298e0410a18 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.250626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856a46a-386d-47d4-a395-1fb48e6d695e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.281794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.282321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.282596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.282859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.295335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.295998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.296271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.296533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.300552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.303117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.303373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.304839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:29.316766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:29.318643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 408/821] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:29.320439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:29.333133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:29.335186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:29.337582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:29.337582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:29.337582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:29.337582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 414/822] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:29.337582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.444339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a30f609-3f90-4ca9-a81f-5541ef55501b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.461563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:29.470797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:29.478739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c3f8d-3643-49d4-8a69-b7c24002bf14 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.486345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.487062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.487338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.487601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.488350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:29.488650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.490140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.505244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 409/823] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:29.506134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.533482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.534510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.534994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.535493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.536749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:29.537290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.539975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.554283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 415/824] 10.222.0.52 () {56 vars in 1124 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:29.554283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.665134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec405a-620b-4643-9604-6864fc4b50e4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.690752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795e31d-c15d-4569-9844-ce6eadf7b9cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.703001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.703705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.704095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.704435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.705163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:29.705730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.707922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.724227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 416/825] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:29.725590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.728901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.729563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.730431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.730669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.731364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:29.731639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.733136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.748097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 410/826] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:29.749440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.758957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:29.801838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:29.862627 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e2cc7-4e3e-46d0-87f9-4ff0fe51a77f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.888568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc27cb5-f426-4d61-97b8-2365d3e27082 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:29.904078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.904758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.905033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.905301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/72d842d8875d4f3f8d44875c410ebf37/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.923487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=72d842d8875d4f3f8d44875c410ebf37, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.926200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.933668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:29.934698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:29.935121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:29.935662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:29.936902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:29.939541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:29.946007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 417/827] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:32:29 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/72d842d8875d4f3f8d44875c410ebf37/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:32:29.947398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:29.973701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 411/828] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:32:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:29.975601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:30.109599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:30.176953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9993ad-178a-4e9f-bf74-a1a50ef21226 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:30.204446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:30.225707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bee24a-0b88-45d4-8f83-b92f75f6a7d8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:30.253957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:30.255038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:30.255507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:30.256050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:30.266658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:30.267397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:30.267679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:30.268059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/0cb99f8c316545e2ba685c87006cbfca/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:30.270024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=0cb99f8c316545e2ba685c87006cbfca, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:30.271494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:30.272146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:30.275107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:30.287725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 412/829] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:32:29 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/0cb99f8c316545e2ba685c87006cbfca/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:30.299373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:30.301372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:30.303445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:30.307293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 418/830] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:30.313712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:30.464431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32d15e-5ee5-41a4-837c-ab3534d906d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:30.509403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:30.510012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:30.510275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:30.510542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:30.511834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:30.512130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:30.513591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82062484-c875-4bc8-b376-f6bdb9e20e10 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:30.527881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 413/831] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:30.535371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:30.680364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da36b-b420-4356-a4ba-cf13b007f86b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:30.734125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:30.740300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:30.740923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:30.741183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:30.741452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:30.742182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=9dc2bb9976f949548bc4743eb297b082) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:30.742482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:30.743932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea63e12-3b0b-446e-948a-d18597ce12ff None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:30.751519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 419/832] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:30 2022] GET /identity/v3/role_assignments?user.id=9dc2bb9976f949548bc4743eb297b082&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:32.378536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec29d1-a1a9-4598-9ce0-be88225ffe14 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:32.379088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec29d1-a1a9-4598-9ce0-be88225ffe14 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:32.379088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec29d1-a1a9-4598-9ce0-be88225ffe14 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:32.380880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 414/833] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:32.386652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2c38e-82fa-43f9-bdee-ef3334daa384 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:32.387154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2c38e-82fa-43f9-bdee-ef3334daa384 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:32.387229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2c38e-82fa-43f9-bdee-ef3334daa384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:32.406921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-98a2c38e-82fa-43f9-bdee-ef3334daa384 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:32.408165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98a2c38e-82fa-43f9-bdee-ef3334daa384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:32.485304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 420/834] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:32.618613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0085e-f46b-4db2-87f9-6592fae36c32 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:32.618613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0085e-f46b-4db2-87f9-6592fae36c32 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:32.618613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0085e-f46b-4db2-87f9-6592fae36c32 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:32.619236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 415/835] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:32.625314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:32.625909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:32.626395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:32.646135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:32.647665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:32.730441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 421/836] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:32.767588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0085e-f46b-4db2-87f9-6592fae36c32 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:32.778891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:32.897811 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0085e-f46b-4db2-87f9-6592fae36c32 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:32.943422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:32.944123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:32.944394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:32.944660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:32.945268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:32.946702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:32.957263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:32.958319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:32.959159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 416/837] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:32.965795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.019665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.038120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:33.094451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee8e9a-a453-403e-88f9-45f52efd28b6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.103982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:33.147009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:33.147697 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.148017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.148290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.149078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:33.149229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:33.150585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb6850-db75-43bf-b788-46d6273df543 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:33.155720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 422/838] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:33.157030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e809db-e368-41f2-b619-963503b97894 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.157338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e809db-e368-41f2-b619-963503b97894 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.157557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e809db-e368-41f2-b619-963503b97894 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.159653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec334d-7c7e-4945-926c-c4ebe72fdf0e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.163260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 423/839] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:33.164643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.164953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.166091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.185729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:33.187018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:33.210750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:33.211571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.212012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.212460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.228123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:33.235065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:33.236905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 417/840] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:33.239607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.285760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 424/841] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:33.286574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.395981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d523f-9ee3-46a9-b125-bbfdefb9c4cb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.403426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:33.448640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:33.456299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:33.457260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.457539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.457813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.459110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:33.459425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:33.461471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47595eb-763b-4dcb-9559-a4dab8af7db4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 418/842] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeadf5-8d60-45fe-98b1-ea6a593a0972 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeadf5-8d60-45fe-98b1-ea6a593a0972 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeadf5-8d60-45fe-98b1-ea6a593a0972 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 419/843] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.483814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.484450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.499580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:33.500862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:33.505827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e76f9-80d2-439e-90e8-d201f839bcdc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.577250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:33.578186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.578672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.579160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.580450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:33.580984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:33.583171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 420/844] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:33.584107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:33.584930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.609252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 425/845] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:33.610638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.749769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244226e-7e1d-4e63-af68-517685b6e204 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.770006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:33.774265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca09c85-1886-47c1-a660-64a723e4251d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.798999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:33.800000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.800287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.800560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.810773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:33.817775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:33.819571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:33.826606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:33.827667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:33.828827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:33.829304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:33.830618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:33.832613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:33.838135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:33.840037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:33.840941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:33.840941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:33.840941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:33.844853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 426/846] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:33.845790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:33.846712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:33.847604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:33.848807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:33.852734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 421/847] 10.222.0.52 () {56 vars in 1132 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:33.859345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:33.963286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ff956-a966-4247-878f-d5fef9fe4952 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:33.981549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89030a7f-3883-425a-b44a-f0a2c57da57e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.000807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.001432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.001712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.001982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.002633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:34.002943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.004353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.018671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 427/848] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:34.019633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.020305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.020579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.020850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.023929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.032538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:34.037972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:34.038961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 422/849] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:32:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:34.040264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.057271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:34.094601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:34.153021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0a3bc-ef8c-4d57-a03d-0914648679b9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.155032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c89af0-3a11-4b2e-a33b-b392d82aab7f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.188463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.189160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.189444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.189720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.195150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.195792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.196068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.196336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.197027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:34.197319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.198019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.199280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.200130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.209037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 428/850] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:34.209037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:34.215568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:34.216827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 423/851] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:34.217860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.312599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:34.326180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82aef74-053e-4a1e-a38e-d19a6385fced None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.369442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99ec31-04fd-45a9-a60c-27998fc5aaaf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.371951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.372896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.373340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.373790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.374541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:34.374881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.376360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.381848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:34.395175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 429/852] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:34.395175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.423435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.424325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.424511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.424779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.425398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:34.425697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.427088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.433560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 424/853] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:34.435446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.569122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8ce88-57ec-4d6e-be8c-db8949da956d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.584732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475d936-a385-4046-9c49-3a573116bc98 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.611014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.611671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.611967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.612228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.626375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.627843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:34.644132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:34.645853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:34.647621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:34.647621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:34.647621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:34.647621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 425/854] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:34.653870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.660178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.660891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.661169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.661494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.662312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:34.662618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.664849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.689296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 430/855] 10.222.0.52 () {56 vars in 1135 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:34.696819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.721390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:34.727411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:34.860736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967c7fb-60e2-4cd1-a54e-e50dfc2b63f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.868395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402cb16-d664-4648-b3dc-2e7e6c140dc1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:34.904990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.906771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.907255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.908090 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.931332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:34.932410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:34.932915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:34.933196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:34.934302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:34.934777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.939028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:34.963333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 426/856] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:34.968722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:34.997511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:34.999693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:35.013571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:35.070267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:35.100573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d913a4c8-393c-4a22-88cd-f03f85065b2f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:35.150945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 431/857] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 28 09:32:35.152649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:35.184231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:35.185172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:35.185590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:35.186233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:35.196928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:35.199630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:35.215049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:35.216323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:35.217612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:35.221289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 427/858] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:32:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:35.231127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:35.340951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de95731-32bc-4296-b9fd-a45ff233daa7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:35.359020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d74f-e4b4-4c83-9cc3-8d983eb2a8ce None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:35.362222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:35.403602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:35.404816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:35.405251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:35.405661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:35.407022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:35.407476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:35.410634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:35.416028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:35.416981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:35.417989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:35.418471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:35.431690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:35.434228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:35.452216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:35.453425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:35.454329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:35.454329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:35.454329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:35.454508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 428/859] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Oct 28 09:32:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:35.454508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:35.462085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:35.462469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 432/860] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:35.465232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:35.617552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a132a-a50a-4096-b481-a68b17cd71eb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:35.628485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754dcbb-94df-476d-9b70-cfc27a437a3b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:35.669170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:35.670309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:35.670901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:35.671387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:35.694773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:35.695815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:35.696308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:35.696783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:35.698928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:35.699480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:35.702785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:35.736952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 433/861] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:35.738639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:35.762796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:35.764926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:35.767393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:35.786213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:35.893840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 429/862] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 28 09:32:35.902663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:35.916023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83478ca-f851-4f77-b2ac-b7e53b073fdc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:35.980930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:35.982569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:35.983350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:35.984160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:35.997872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.000978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:36.025678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:36.028431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:36.036528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 434/863] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:36.041006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978b993f-a784-4fe8-b323-53ea24d138d9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:36.041941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:36.043442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:36.092596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:36.093245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.093523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.093789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.109117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.113407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2f796797-14ef-45cf-a9ff-e320664ecc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:36.132404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:36.134262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:36.136565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:36.136565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:36.136565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:36.138926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 430/864] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:36.144351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a4a6-09df-48e9-acd6-c2e9b523a319 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.144351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a4a6-09df-48e9-acd6-c2e9b523a319 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.144351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1a4a6-09df-48e9-acd6-c2e9b523a319 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.144351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 431/865] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:36.144351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a4a6-09df-48e9-acd6-c2e9b523a319 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:36.188551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:36.216495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf86b-2ab9-4b93-88e8-77b3f9e2e0d3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:36.260190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:36.260721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.261101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.261374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.261739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1a4a6-09df-48e9-acd6-c2e9b523a319 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:36.262084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:36.262414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.263885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a68e3-c1ce-4fe5-9463-914c1a980ab3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.278272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 435/866] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:36.279328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.279611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.279910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.305126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:36.306349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:36.320427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:36.321049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.321329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.321603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.322258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:36.322561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.324115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.337623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 432/867] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:36.338327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:36.364930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:36.410395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 436/868] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:36.416014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:36.482617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c038a-5dee-4c01-be43-5eddd9326153 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:36.500377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:36.533572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:36.534272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b2f05a-d82e-4f88-9ef0-310ae7b7f45f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:36.534632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.535060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.535481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.536639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=0cb99f8c316545e2ba685c87006cbfca) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:36.538035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.539721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.554309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 433/869] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:36 2022] GET /identity/v3/role_assignments?user.id=0cb99f8c316545e2ba685c87006cbfca&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:36.579818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:36.580809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.581247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.581676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.590923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.592815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:36.606380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:36.607527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:36.608692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:36.610944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 437/870] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:36.616952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:36.720435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:36.747730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc4b0b-0743-4241-80bd-ecd8972e499c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:36.788430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:36.789358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:36.789641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:36.789914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:36.790617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:36.790922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:36.792444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:36.805887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 434/871] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:36.812734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:36.898154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:37.003129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723fd86-7c41-48f6-915e-b02d98983c4f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:37.067244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:37.078866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:37.080245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:37.080859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:37.081459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:37.082971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=72d842d8875d4f3f8d44875c410ebf37) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:37.083408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:37.087618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af43c-930a-404a-9f9b-53e7d9363870 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:37.097493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23912d-3a12-4314-84cf-48ff1ab6f161 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:37.103812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 438/872] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:36 2022] GET /identity/v3/role_assignments?user.id=72d842d8875d4f3f8d44875c410ebf37&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:37.195406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e64b00e-3f65-4375-b3b2-c585fd975b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:37.196504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64b00e-3f65-4375-b3b2-c585fd975b5f None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:37.197055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64b00e-3f65-4375-b3b2-c585fd975b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:37.197538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64b00e-3f65-4375-b3b2-c585fd975b5f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:37.199325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64b00e-3f65-4375-b3b2-c585fd975b5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:37.201984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e64b00e-3f65-4375-b3b2-c585fd975b5f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:37.221109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:37.236288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 435/873] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:32:36 2022] POST /identity/v3/services => generated 251 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:37.630204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5035-d019-4ba4-9fc3-e0fde1140e5d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:37.630516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5035-d019-4ba4-9fc3-e0fde1140e5d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:37.630792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5035-d019-4ba4-9fc3-e0fde1140e5d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:37.632454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 439/874] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:37.638777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab2ef4-5dbf-4cbc-ae6b-9499f1f7b372 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:37.639320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab2ef4-5dbf-4cbc-ae6b-9499f1f7b372 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:37.639320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab2ef4-5dbf-4cbc-ae6b-9499f1f7b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:37.659172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eaab2ef4-5dbf-4cbc-ae6b-9499f1f7b372 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:37.660421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eaab2ef4-5dbf-4cbc-ae6b-9499f1f7b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:37.743295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 436/875] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:38.012924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5035-d019-4ba4-9fc3-e0fde1140e5d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:38.170786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5035-d019-4ba4-9fc3-e0fde1140e5d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:38.217883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afc0fd-fd19-40cc-a9fd-49cb8b747b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:38.219445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afc0fd-fd19-40cc-a9fd-49cb8b747b75 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:38.220012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afc0fd-fd19-40cc-a9fd-49cb8b747b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:38.220775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afc0fd-fd19-40cc-a9fd-49cb8b747b75 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:38.222224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afc0fd-fd19-40cc-a9fd-49cb8b747b75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:38.225197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afc0fd-fd19-40cc-a9fd-49cb8b747b75 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:38.250206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 440/876] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:32:38 2022] POST /identity/v3/services => generated 269 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:38.934277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad09872-649a-4a44-a6b1-4b733d3a8c8c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:38.934869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad09872-649a-4a44-a6b1-4b733d3a8c8c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:38.934869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad09872-649a-4a44-a6b1-4b733d3a8c8c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:38.936487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 437/877] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:38.942467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:38.942773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:38.943012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:38.963176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:38.965046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:39.022466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:39.048939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 441/878] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:39.319007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad09872-649a-4a44-a6b1-4b733d3a8c8c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:39.432327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad09872-649a-4a44-a6b1-4b733d3a8c8c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:39.458527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:39.472952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:39.473598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.473871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.474135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.474722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:39.476233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:39.486536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:39.487657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:39.488609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:39.492115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 438/879] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:39.496888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:39.607014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6660643-e7c0-4d47-871b-35cb630e3a3c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:39.648030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:39.648656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.648930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.649199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.649834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:39.650132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:39.652099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab89fc0-55f3-4fa1-ae47-aa5e404147ac None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:39.657794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 442/880] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:39.663477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:39.739383 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711412-ed63-4db1-a868-66355277936a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.739730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711412-ed63-4db1-a868-66355277936a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.739969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711412-ed63-4db1-a868-66355277936a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.741467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 443/881] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:39.747773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff50e-4acc-472f-8515-419d4859da6e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.748078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff50e-4acc-472f-8515-419d4859da6e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.748335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ff50e-4acc-472f-8515-419d4859da6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.769777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2d7ff50e-4acc-472f-8515-419d4859da6e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:39.771114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2d7ff50e-4acc-472f-8515-419d4859da6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:39.786547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36339178-1a24-480a-91ae-a30e6aa65a2c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:39.850062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:39.851216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.851845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.852371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.853626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:39.854575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:39.855625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:39.859259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77225918-8398-4dfa-ba4e-62392333def5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:39.875976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 439/882] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:39.900532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 444/883] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:39.903731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4111bb-d36d-4b8e-8e94-5dacddbbcde6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.904360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4111bb-d36d-4b8e-8e94-5dacddbbcde6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.904934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4111bb-d36d-4b8e-8e94-5dacddbbcde6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.907814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 440/884] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:39.914300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:39.915050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:39.915859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:39.952937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:39.955302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:40.083043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 445/885] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:40.200580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4111bb-d36d-4b8e-8e94-5dacddbbcde6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:40.275787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:40.355586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4111bb-d36d-4b8e-8e94-5dacddbbcde6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:40.363622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:40.428374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:40.430036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.430886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.431681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.434065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:40.437166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:40.455601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:40.457521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:40.459028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:40.471872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 441/886] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:40.475148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:40.485535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818a8cbb-f728-41dd-a7f0-7e381244214d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:40.500170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:40.527735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:40.528354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.528614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.531780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.542988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:40.548597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:40.550467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:32:40.552102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:40.559807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 446/887] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:40.559807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:40.653305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb172e-8447-4018-883e-ea22b925d78f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:40.684518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:40.685656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e6e92-ae74-48fe-a735-bc23870fdc6a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:40.720582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:40.721515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.721909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.722299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.723243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:40.723696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:40.725919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb27eb-c05b-4048-9b57-5f0f61ee7a86 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:40.735096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 442/888] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:40.737256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4cd0-149c-4fe9-8733-9322ae03080c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.737732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4cd0-149c-4fe9-8733-9322ae03080c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.738174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd4cd0-149c-4fe9-8733-9322ae03080c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.740878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 443/889] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:40.742245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd4cd0-149c-4fe9-8733-9322ae03080c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:40.747565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:40.749025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.750840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.751326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.752638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:40.753177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:40.755887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e86656c-7c24-4056-9a9c-7aab875112f8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:40.780001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 447/890] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:40.783324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.783824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.784834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.814751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:40.816658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:40.850261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:40.902399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd4cd0-149c-4fe9-8733-9322ae03080c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:40.943919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 448/891] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:40.943919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:40.967129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:40.968338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:40.968679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:40.969021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:40.969909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:40.970603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:40.973943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:40.984002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 444/892] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:40.992709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.088872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:41.143872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c0f5-4fe4-4e2c-bf4f-9551f9afa426 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.167895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afa24e-1459-4306-99af-47fd0a864c57 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.205948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.207005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.207185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.207931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.208089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.208089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.208382 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.208561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.209011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] RBAC: Adding query filter params (interface=public, service_id=df94c325d5794935969c297cb0ab8501, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:41.209315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.211192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.222051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 445/893] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/endpoints?service_id=df94c325d5794935969c297cb0ab8501&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:41.223009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.229329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.232611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:41.262616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:41.266322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:41.269760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:41.269760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:41.269760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:41.269760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 449/894] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:41.281289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.295016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:41.367374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccb35b-dff7-4526-bc3a-9f4372f6cb5f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.379082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:41.407211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.407871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.408209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.408479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.409142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.410550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.417540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 446/895] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:32:41.418387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.473551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a26e6e-5ce7-40b1-ad9d-ecc34315b8e2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.540947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.542955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.543554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.544055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.545170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:41.545776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.547972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.568238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be44602-90a1-4d97-bf80-645c45cc473e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.573641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 450/896] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:41.575891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.616565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.618060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.618703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.619268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.620580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.622965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:41.636419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:41.637792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:41.639018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:41.642482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 447/897] 10.222.0.52 () {56 vars in 1132 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:41.651022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.689802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:41.724998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:41.739330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818ee1e-ef17-4300-9f0c-88da965d2d41 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.778245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.778522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.778787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.779049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.787816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.789241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:41.802246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:41.804456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:41.806722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 451/898] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:41.809456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.857252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882b4e5-3743-4dba-8ac9-814b5a7c400a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:41.923442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:41.924729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:41.925292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:41.925943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:41.927200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:41.927854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:41.930653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:41.940739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 448/899] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:41.947298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:41.985495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15678d-d41a-420b-862c-7d40a158933d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:42.037029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:42.055990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:42.057600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.058434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.060176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.062273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:42.063128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:42.067017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:42.114018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 452/900] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:42.115677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:42.157151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:42.163516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae11cd-e42c-46c1-9b60-35a00fd66b4a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:42.228437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:42.229591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.230143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.230700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.231991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:42.232595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:42.235223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:42.250210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 449/901] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:42.252036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:42.343504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678a024-f06e-4f3f-bc1e-3e3259390ba9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:42.397541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:42.398924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.399603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.400625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.444131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaead19-d5a6-43da-9c05-acb6adec8030 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:42.493351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:42.498773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:42.499573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.499879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.500168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.500969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] RBAC: Adding query filter params (interface=public, service_id=f740ad4d4e4746cc8ff891a8d314baab, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:42.501253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:42.502736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:42.511970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 450/902] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:32:42 2022] GET /identity/v3/endpoints?service_id=f740ad4d4e4746cc8ff891a8d314baab&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:42.517401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:42.529887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:42.533173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:42.555727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:42.659869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e88b4-4710-4d90-9e14-1967674718ec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:42.703491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f972ba-84eb-467c-897e-194ae01525a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:42.704138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f972ba-84eb-467c-897e-194ae01525a4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.704416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f972ba-84eb-467c-897e-194ae01525a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.704693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f972ba-84eb-467c-897e-194ae01525a4 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.705305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f972ba-84eb-467c-897e-194ae01525a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:42.706699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f972ba-84eb-467c-897e-194ae01525a4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:42.720996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 451/903] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:42 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:32:42.720996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631aac-7a3c-4500-b578-457aca351456 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.720996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631aac-7a3c-4500-b578-457aca351456 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.720996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf631aac-7a3c-4500-b578-457aca351456 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.721792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 452/904] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:42.723418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 453/905] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:42 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:32:42.723713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.724053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.724328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.724997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:42.744328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:42.745645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:42.866824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:42.882377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 453/906] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:42.898729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197e0a0-c98a-4123-a277-881c3728a732 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:42.963457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:42.964557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:42.965022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:42.965616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:42.984533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:42.987183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:43.017146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:43.018958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:43.020704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:43.020704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:43.020704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:43.020704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 454/907] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:43.028840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:43.072937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:43.155551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:43.237579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f869fe1-4418-4a04-976b-ee70e4cf61fe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:43.311689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:43.312754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:43.313211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:43.313673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:43.314813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:43.315396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:43.318134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:43.347015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 454/908] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:43.352456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:43.355144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3edc1b-2918-49d9-9da0-8c1db2277668 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:43.427387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:43.428473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:43.428925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:43.429364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:43.430472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:43.433014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:43.451111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:43.468576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 455/909] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:32:43 2022] POST /identity/v3/services => generated 241 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:32:43.529753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:43.568119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344047f-5570-4391-916a-be5ebd3e33f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:43.624075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:43.624723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:43.624994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:43.625260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:43.633118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:43.634593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:43.648407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:43.650549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:43.653269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 455/910] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:43.656756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:43.787210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65157a5e-8ed0-45a4-aadd-a5b4bc66428f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:43.827799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:43.831058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:43.831058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:43.831058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:43.831058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:43.831058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:43.833328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a54f6-d3f4-48dc-a6aa-b6c5745b2fa7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:43.849105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 456/911] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:43.855292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:43.961356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644fb53-8200-4c30-9948-d4cc66cb4985 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:43.961773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644fb53-8200-4c30-9948-d4cc66cb4985 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:43.962055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644fb53-8200-4c30-9948-d4cc66cb4985 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:43.963414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 457/912] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:43.968490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff98fd3-f556-4011-b240-509c445ba7fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:43.970829 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:43.972203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:43.972608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:43.996906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:43.998925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:44.006244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:44.006866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:44.007137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:44.007401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:44.008171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] RBAC: Adding query filter params (role.id=9f80f81454c74975ad8943f88fe0e487, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=72d842d8875d4f3f8d44875c410ebf37) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:44.008473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:44.010610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:44.021220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 456/913] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:43 2022] GET /identity/v3/role_assignments?user.id=72d842d8875d4f3f8d44875c410ebf37&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9f80f81454c74975ad8943f88fe0e487 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:44.076854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:44.099485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 458/914] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:44.606164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:44.610505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:44.803466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695d9d6-c8e3-4563-8cf8-7d480786028c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:44.867306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:44.868390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:44.868879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:44.869346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:44.871031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:44.873790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:44.892012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:44.893962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:44.895656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 457/915] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:44.908268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:45.102257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a328b-dc19-4e65-a39f-8cdb54dfa6e6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:45.169965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:45.170969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:45.171420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:45.171892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:45.173012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:45.173542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:45.176233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73786ae-5364-48f6-b177-c1ffcf827726 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:45.186752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 459/916] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:45.196359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:45.200426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe10ac-1abc-4969-9cd9-d8ed08f674d5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:45.200774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe10ac-1abc-4969-9cd9-d8ed08f674d5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:45.200959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe10ac-1abc-4969-9cd9-d8ed08f674d5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:45.202607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 460/917] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:45.208221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:45.208480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:45.208800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:45.232243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:45.233483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:45.292727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:45.320347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:45.350714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 461/918] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:45.360004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703047d-9f52-4a94-a115-3d448cb5bd9f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:45.411451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:45.412005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:45.412280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:45.412546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:45.413192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:45.413495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:45.414912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:45.421669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 458/919] 10.222.0.52 () {56 vars in 1114 bytes} [Fri Oct 28 09:32:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:45.431189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:45.569490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf690829-dbcd-49a4-9a1b-33a87510d8cc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:45.606962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:45.607583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:45.607888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:45.608155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:45.608842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:45.610221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:45.635207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 462/920] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:32:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:32:45.868555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:46.103002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8785e26e-d911-44fe-97bc-f0a43a2c3bc4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:46.172139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:46.192861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:46.193951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.194427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.194916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.196013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:46.199223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9e208f18-c14a-4bb4-91f4-25fa21feb735 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:46.217368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:46.218397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 28 09:32:46.219297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:46.220073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 459/921] 10.222.0.52 () {56 vars in 1132 bytes} [Fri Oct 28 09:32:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:46.229569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:46.372294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:46.538932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67111e02-da35-478c-88c8-16c4d06f5478 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:46.606632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068c51-ca07-4b33-9296-c53a6382b641 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.607056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068c51-ca07-4b33-9296-c53a6382b641 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.607549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74068c51-ca07-4b33-9296-c53a6382b641 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.617044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 460/922] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:46.621603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.622461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.623422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.626950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:46.627624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.627940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.628159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.628802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:46.629142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:46.630485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:46.637632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 463/923] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:46.642964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:46.653478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:46.655034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:46.738364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 461/924] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:46.784327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789d946-b6a1-4dd4-9ad6-3cca5d4419e9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:46.788610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:46.824335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:46.825270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.825725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.826166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.827004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:46.827612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:46.829174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fc0c25-f7d4-48b2-a24c-055468d490a3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:46.835154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 464/925] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:46.842043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:46.896025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591108c-ad87-4aa5-9c5f-47bcc597bc8d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.896298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591108c-ad87-4aa5-9c5f-47bcc597bc8d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.896577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591108c-ad87-4aa5-9c5f-47bcc597bc8d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.898425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 465/926] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:46.904286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:46.904567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:46.904830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:46.924309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:46.925813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:46.968868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca5c1a-0e8a-468b-ad66-f5694cc78650 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:47.002427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:47.026690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:47.027317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:47.027593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:47.027890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:47.028575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:47.030202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:47.057903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 466/927] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:47.078141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 462/928] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:32:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:32:47.123896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:47.322796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:47.333772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ea3d8-dfa3-4d5b-aa97-13bbbfc761f6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:47.410927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:47.412042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:47.412528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:47.413003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:47.414128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:47.416744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:47.431628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:47.433289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:47.434146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:47.435578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 467/929] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:47.442000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:47.535940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:47.583663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64687579-259f-4c8b-98d4-75e447d08af8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:47.632864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:47.633407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:47.633685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:47.633955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:47.634589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:47.634888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:47.637912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:47.647074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 463/930] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:47.654145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:47.679035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602de8c-848d-4579-8c77-d149fa6a2367 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:47.714076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:47.737469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:47.738388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:47.738806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:47.739211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:47.751832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:47.757809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:47.758788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:32:47.759785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:47.763064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 468/931] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Oct 28 09:32:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:47.774867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:47.841028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e176c-7c94-43bb-875d-44c9443a4d96 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:47.876153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:47.889718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:47.890236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:47.890512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:47.890781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:47.892519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:47.892864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:47.894290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:47.903934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 464/932] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:47.908862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:47.953265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615a43-fec7-4528-bab3-db7b441991a3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.021370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.022011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.022283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.022559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.023615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:48.024174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.026298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.044127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 469/933] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:32:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:48.051347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:48.074078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81080e-c8fb-42b6-a565-3a7e8e6a2820 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.080193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:48.138424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.139460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.140032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.140528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.141451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] RBAC: Adding query filter params (interface=public, service_id=198cd2c3d99f47ce83c0409675f3f3a5, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:48.141980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.143642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.159821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 465/934] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:32:47 2022] GET /identity/v3/endpoints?service_id=198cd2c3d99f47ce83c0409675f3f3a5&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:48.159821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:48.162283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:48.199054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ed07a-6400-4279-aece-95ea177935ae None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.261668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.262764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.263223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.263696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.275474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03eabe5-f517-4734-9fbb-5fe8b008f0e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.286461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.289197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.314909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6458a-896b-4880-8e4d-d0118680284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.316184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6458a-896b-4880-8e4d-d0118680284b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.316687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6458a-896b-4880-8e4d-d0118680284b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.317224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6458a-896b-4880-8e4d-d0118680284b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.318127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6458a-896b-4880-8e4d-d0118680284b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:48.318434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:48.319376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:48.320357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:48.320357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:48.320357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:48.322692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6458a-896b-4880-8e4d-d0118680284b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.326662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 470/935] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:48.330948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:48.339942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 466/936] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:48 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:32:48.458484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1552c6-5874-4365-8231-eb0c32bf9d51 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.458824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1552c6-5874-4365-8231-eb0c32bf9d51 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.459038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1552c6-5874-4365-8231-eb0c32bf9d51 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.461035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 467/937] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:48.467356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.467682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.467945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.493759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:48.496121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:48.499246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e5d1c-f170-4eb2-aefd-71bef1d754fe None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.529749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:48.538337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:48.546455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.547074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.547345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.547610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.548288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:48.548650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.550039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.563159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 471/938] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:48.569058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:48.601435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 468/939] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:48.681310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833443-91b4-4c65-a522-12f1eb6ff491 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.718368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.718925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.719197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.719464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.727301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.728770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:48.741286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:48.742307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:48.743287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 472/940] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:32:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:48.748566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:48.918737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0b4e8-851d-42b3-803c-a1b29c2c0016 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:48.923608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:48.958723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:48.959343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:48.959620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:48.959926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:48.960932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:48.961238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:48.962825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:48.980740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:48.991781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 469/941] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Oct 28 09:32:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:48.992789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:49.017302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:49.049516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbab90-b81e-4d8f-98dc-4148866d7943 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.095041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.096125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.096611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.097081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.097991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.099660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.102600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING keystone.server.flask.common [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 28 09:32:49.115495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc1656-86a6-4bf2-ba51-1f4efaf6afea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.124481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.sql.core [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 28 09:32:49.124481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 09:32:49.124481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [parameters: {'id': '0d83188d5b1b4126bcb04e230042e6ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 28 09:32:49.124481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81610) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d83188d5b1b4126bcb04e230042e6ec.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d83188d5b1b4126bcb04e230042e6ec. Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application conn.query(q) Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 09:32:49.132922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result.read() Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 28 09:32:49.133791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application next(self.gen) Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application yield resource Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application next(self.gen) Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise self.value Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application session.commit() Oct 28 09:32:49.134792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.session.flush() Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application compat.raise_( Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 09:32:49.135852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 09:32:49.137491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application conn.query(q) Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application result.read() Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 09:32:49.138287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [parameters: {'id': '0d83188d5b1b4126bcb04e230042e6ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.140507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:49.142442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0d83188d5b1b4126bcb04e230042e6ec. Oct 28 09:32:49.144535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.152220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 473/942] 10.222.0.52 () {60 vars in 1140 bytes} [Fri Oct 28 09:32:48 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:49.154415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:49.169098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.169961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.170373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.170740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.183373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.185177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:49.202436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:49.203687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:49.204780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:49.204780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:49.204780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:49.211056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 470/943] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:49.212307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:49.303221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:49.321133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab900a4e-8e00-48a5-812b-4e5766d62c3c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.334328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:49.346819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fe81a-87fa-4eea-bb5e-e340f22f3bba None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.373162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.373808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.374082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.374339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.382166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.384379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.398131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:49.399162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:49.400464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:49.402363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:49.403622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.404341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.404931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.405800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 474/944] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:49.406358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:49.406950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.407055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:49.409239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.430774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 471/945] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:49.433367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:49.536962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba34f68-410f-410b-80cf-740e2884cc23 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.569375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6995274-d69b-4036-bb0c-a67e7101a428 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.578615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.579235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.579505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.579788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.580471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:49.580826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.582381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352ee6e-1d43-40ac-951d-718b8af68b9f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.607402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 475/946] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:49.615359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9a436-9c03-4143-bf9e-ce5d22582d98 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.615696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9a436-9c03-4143-bf9e-ce5d22582d98 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.615939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9a436-9c03-4143-bf9e-ce5d22582d98 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.617557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 476/947] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:49.624536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.624867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.625076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.631351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:49.640837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.641510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.641784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.642051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.649123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:49.650405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.652039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:49.652498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:49.668607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:49.670034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:49.670938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 472/948] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:49.676210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:49.691110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:49.768589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 477/949] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:49.832492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31737f9-9fc2-4dfa-b6fb-c5d37c03cfbe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:49.880643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:49.881633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:49.882130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:49.882618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:49.883923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:49.884485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:49.887331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:49.902131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 473/950] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:49.908346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:50.035617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2a52-fd9e-4169-a216-da480290fa91 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:50.049016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:50.085880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:50.086475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:50.086745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:50.086994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/72d842d8875d4f3f8d44875c410ebf37/roles/9f80f81454c74975ad8943f88fe0e487` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:50.091206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=72d842d8875d4f3f8d44875c410ebf37, role_id=9f80f81454c74975ad8943f88fe0e487)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:50.093767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:50.113042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:50.116244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 478/951] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:32:49 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/72d842d8875d4f3f8d44875c410ebf37/roles/9f80f81454c74975ad8943f88fe0e487 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:32:50.200693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76333ade-7769-43c3-aec4-42e164b77eec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:50.252739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37204048-898b-483a-994b-f1f633178160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:50.253399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37204048-898b-483a-994b-f1f633178160 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:50.253716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37204048-898b-483a-994b-f1f633178160 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:50.254070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37204048-898b-483a-994b-f1f633178160 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:50.254802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37204048-898b-483a-994b-f1f633178160 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:50.256457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37204048-898b-483a-994b-f1f633178160 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-37204048-898b-483a-994b-f1f633178160 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:50.272481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:50.273693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:32:50.274599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:50.275380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 474/952] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:50.282858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:50.311565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:50.445766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f983a8-e65b-4107-b30f-5d4bc31d33bf None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:50.487249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:50.487876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:50.488157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:50.488401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:50.489036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:50.489332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:50.490696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0eb697-dc07-4d55-8ee8-a94c24a70587 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:50.496649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 479/953] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:50.504879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37204048-898b-483a-994b-f1f633178160 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:50.664352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37204048-898b-483a-994b-f1f633178160 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:50.686961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:50.708058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:50.708621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:50.708991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:50.709173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:50.709838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:50.710142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:50.711885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:50.714872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dd404-07e3-414a-b4c2-1d1a2bdbcb4b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:50.715241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dd404-07e3-414a-b4c2-1d1a2bdbcb4b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:50.715439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323dd404-07e3-414a-b4c2-1d1a2bdbcb4b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:50.721585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 480/954] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:50.722170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 475/955] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:50.723556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:50.723897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:50.724183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:50.746869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:50.748367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:50.843891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 481/956] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:51.112239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:51.228692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7166d30-4276-45bb-99d8-a46f5e05091a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:51.266645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:51.267298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:51.267578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:51.267931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:51.268574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:51.269975 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:51.280149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:51.283978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:51.287312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 476/957] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:51.294508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:51.312793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:51.481767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e53487-7979-4f50-982d-fdfd78cc0fc1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:51.536371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:51.537010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:51.537284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:51.537553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:51.538211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:51.538509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:51.540075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:51.551532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 482/958] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:51.552991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:51.617422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:51.673803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cc374-c52a-42f9-ab30-e6a07a3befd7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:51.717423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:51.718184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:51.718464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:51.718721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:51.719391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:51.719694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:51.721212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf7ca3-f4e0-419c-ab04-3dec713d3596 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:51.732065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 477/959] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:51.734542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:51.823134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:51.852708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8290-ca5c-4742-ad86-7bfc78fa5b1a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:51.891616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c082c-d425-417d-9d71-195d0e86c087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:51.892317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c082c-d425-417d-9d71-195d0e86c087 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:51.892580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c082c-d425-417d-9d71-195d0e86c087 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:51.892843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c082c-d425-417d-9d71-195d0e86c087 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:51.893609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c082c-d425-417d-9d71-195d0e86c087 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:51.894997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c082c-d425-417d-9d71-195d0e86c087 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:51.933864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 483/960] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:32:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:52.490237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f33d7-fbd4-4466-8693-705023e1e520 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:52.490981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f33d7-fbd4-4466-8693-705023e1e520 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:52.490981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f33d7-fbd4-4466-8693-705023e1e520 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:52.492431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 478/961] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:52.498093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:52.498687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:52.498687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:52.518295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:52.519489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:52.556940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32feb3-beee-48d9-9a31-d711120629f6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:52.556940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32feb3-beee-48d9-9a31-d711120629f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:52.556940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32feb3-beee-48d9-9a31-d711120629f6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:52.557957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 479/962] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:52.563770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:52.564398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:52.564886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:52.593479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:52.595506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:52.605718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 484/963] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:52.713615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 480/964] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:52.879276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:52.943275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:52.996648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80f22-f3cc-4c09-807b-5d99ce395788 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.013818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.045565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.046587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.046874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.047143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.065889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:32:53.070664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:32:53.072575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:53.073757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:32:53.074621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:53.075295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 485/965] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Oct 28 09:32:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:53.080429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.169837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b679e1-2ebd-48a8-93ab-631a2e72dd5c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.210419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.211413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.211621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.211940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.212601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:53.214685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:53.217362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING keystone.server.flask.common [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 28 09:32:53.231438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 28 09:32:53.231438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 28 09:32:53.231438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [parameters: {'user_id': '5a997f02304440f3a6edae4b68645369', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 28 09:32:53.231438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 28 09:32:53.231886 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8dcfac-d17a-4837-a7f4-771e26aca477 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8124e4f8-e1b8-4dfe-94ce-515a49165b4a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Oct 28 09:32:53.251043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 09:32:53.252023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 28 09:32:53.253006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 28 09:32:53.253923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 28 09:32:53.254816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 28 09:32:53.256908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'user_id': '5a997f02304440f3a6edae4b68645369', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:53.259653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 28 09:32:53.261398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 481/966] 10.222.0.52 () {60 vars in 1131 bytes} [Fri Oct 28 09:32:53 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc1ab6-e1b8-429b-8711-8980a08080f1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc1ab6-e1b8-429b-8711-8980a08080f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.262412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc1ab6-e1b8-429b-8711-8980a08080f1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.263292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 482/967] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:53.263292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc1ab6-e1b8-429b-8711-8980a08080f1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.263777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:53.287426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.288087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.288361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.288631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.289318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:53.289691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:53.291135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e841af1-2555-4bb8-9d5d-a09e2af56ca9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:53.301317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 486/968] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:32:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:53.302320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.302621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.302898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.324079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:53.325322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:53.398985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:53.405502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc1ab6-e1b8-429b-8711-8980a08080f1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.416080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 487/969] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:32:53.417013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.477299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.478877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.479812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.480494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.497278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:53.501832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:53.526936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aec180-ad66-42c1-b965-6f110905f2ca None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:53.529965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:53.532141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 28 09:32:53.534385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:53.537791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:53.544378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 483/970] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:32:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:53.551433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.570366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.571020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.571309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.571584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.584064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:53.585813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:53.602872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:53.605126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:53.607245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:53.607245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:53.607245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:53.607245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 488/971] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:53.610210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.721724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dac47-55f3-4b9f-a12e-0c021df46ee5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.762871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.763779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.764078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.764342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.765113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:53.765411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:53.766791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:53.772387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88b0e4-f321-42ec-83f1-b23b230f8dc2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:53.787957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 489/972] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:53.787957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.817414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:53.835289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:53.836773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:53.837264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:53.837728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:53.839051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:53.839560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:53.842252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:53.914385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 484/973] 10.222.0.52 () {56 vars in 1101 bytes} [Fri Oct 28 09:32:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:53.915933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:53.950326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:53.962284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08b1d5-866c-435b-9a13-6456202750aa None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:54.034047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:54.034951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:54.035395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:54.035858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:54.036809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:54.038477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:54.058599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 490/974] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:32:53 2022] POST /identity/v3/services => generated 242 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:32:54.124515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d747285-64e8-43d0-a4e5-13251631927b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:54.216782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:54.218026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:54.218501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:54.219053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:54.234678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:54.237827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:32:54.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:54.264767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 28 09:32:54.265706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:54.271530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 485/975] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:32:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:54.276959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:54.442503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:54.476447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc81e5-6ff6-43b0-9ab6-837c8087c99d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:54.542271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:54.543287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:54.543786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:54.544257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:54.545611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:54.546143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:54.548957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6a79f-e299-4445-858e-fa9e0063e1f6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:54.587837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 491/976] 10.222.0.52 () {56 vars in 1149 bytes} [Fri Oct 28 09:32:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:54.589471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:54.731657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:54.785722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa0ad13-f12a-4cba-ba13-152224bcbad8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:54.847464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:54.848098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:54.848427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:54.848700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:54.898295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e764f17-4f64-418e-8d0f-4bde4275c52f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:54.898562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e764f17-4f64-418e-8d0f-4bde4275c52f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:54.898981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e764f17-4f64-418e-8d0f-4bde4275c52f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:54.900563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 492/977] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:32:54.906124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:54.906386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:54.906732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:54.927326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:54.928560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:54.953654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:54.956847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:55.043716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 493/978] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:55.078526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:55.134638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 486/979] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:32:54 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:32:55.140230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:55.222461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:55.262871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73271da8-c2e6-44ac-b668-10ec2fba3153 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:55.303430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:55.304036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:55.304355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:55.304667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:55.316594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:55.318052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:32:55.335427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:55.336978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:32:55.340021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:32:55.340021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:32:55.340021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:55.347810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 494/980] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:55.348148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:55.462477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7da641-aad6-4681-ab18-4a3716d394c3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:55.486391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:55.509284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:55.509924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:55.510746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:55.511181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:55.511874 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:55.512176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:55.513588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:55.528208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 487/981] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:55.533796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:55.580170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:55.655488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2645be-cd54-481b-8270-4e13d8da8931 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:55.694099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:55.708307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee4b23f-ad3c-49bd-aef3-860a42747c02 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:55.718112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:55.718920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:55.719318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:55.719717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:55.720486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:55.721982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:55.733950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:55.737199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:32:55.738183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 495/982] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:32:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:55.745772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:55.773381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:55.774686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:55.775128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:55.775563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:55.788991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:55.791214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:32:55.810609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:55.812748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:55.815202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 488/983] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:32:55.819688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:55.932219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11cb76-f745-48ef-b470-5020c15e3277 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:55.966273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:55.975535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de27d7b-2d54-4f90-be10-b9a6abe13b57 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:55.994735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:55.995900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:55.996300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:55.997199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:55.998014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:55.998440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:56.000194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:56.012513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 496/984] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:56.018968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:56.025456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:56.026981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.027425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.027893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.029065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:56.029539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:56.032070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:56.060785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 489/985] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:32:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:56.063535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:56.119026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:56.204359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602f3cb-8734-43ad-96d4-b92b7330479d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:56.241119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6c818-a5f3-4bdf-aff6-c5c36a299beb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:56.255170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:56.256271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.256711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.257148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.258199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:56.258660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:56.261631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:56.270950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 497/986] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:32:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:56.282756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:56.302282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:56.303052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.303457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.303950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.304808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] RBAC: Adding query filter params (role.id=9f80f81454c74975ad8943f88fe0e487, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=72d842d8875d4f3f8d44875c410ebf37) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:56.305231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:56.306862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dba3b3-f490-41a8-8591-8e08a542e0ce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:56.324099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 490/987] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:32:56 2022] GET /identity/v3/role_assignments?user.id=72d842d8875d4f3f8d44875c410ebf37&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9f80f81454c74975ad8943f88fe0e487 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:56.493526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa008e-6365-4cce-8721-e63d03d730bb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:56.521288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:56.559571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:56.560642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.561138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.561565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.562432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3c44b-8530-4e62-8e1d-2c704e6c2537 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.562772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] RBAC: Adding query filter params (interface=admin, service_id=198cd2c3d99f47ce83c0409675f3f3a5, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:56.562873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3c44b-8530-4e62-8e1d-2c704e6c2537 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.563267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc3c44b-8530-4e62-8e1d-2c704e6c2537 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.563363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:56.565599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 491/988] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:56.566011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:56.571709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.572022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.572301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.575663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 498/989] 10.222.0.52 () {56 vars in 1249 bytes} [Fri Oct 28 09:32:56 2022] GET /identity/v3/endpoints?service_id=198cd2c3d99f47ce83c0409675f3f3a5&interface=admin®ion_id=RegionOne => generated 193 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:56.585353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:56.593405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:56.595048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:56.614973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:56.694434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 492/990] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:32:56.778558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69078-ec14-4eef-9a04-67c18734c0b3 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:56.847871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:56.848875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:56.849330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:56.849789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:56.850846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:56.853438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:56.871780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 499/991] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:56 2022] GET /identity/v3/services => generated 1727 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:32:56.968474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:57.091093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19656d38-88fc-4326-8c9f-2ba7c3785b2e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:57.142857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:57.143922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:57.144408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:57.144898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:57.145716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:57.147359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:32:57.157156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:32:57.159065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:32:57.160760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 493/992] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:32:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:32:57.164910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:57.215975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:57.286762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebfddd-2806-4d55-a0bf-7ad0299b9694 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:57.326634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:57.327246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:57.327517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:57.327810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:57.328457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:57.328752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:57.330136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:57.336418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 500/993] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:57.343344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:57.371865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:57.529682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25556-3fc6-4714-bb92-7f651de4da79 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:57.589297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:57.590187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:57.590590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:57.591057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:57.592214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:57.592732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:57.595186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:57.607778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 494/994] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:32:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:32:57.610095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:57.729101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aed69a-6a25-4ef5-9a56-41606436105c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:57.731894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:57.769977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:57.770592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:57.770874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:57.771127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:57.771859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] RBAC: Adding query filter params (interface=public, service_id=ea359789c43443b28c8695a48eba219a, region_id=RegionOne) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:32:57.772167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:57.773598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad3c17-1dcc-49d6-a86f-08e7dc54b256 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:57.780455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 501/995] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:32:57 2022] GET /identity/v3/endpoints?service_id=ea359789c43443b28c8695a48eba219a&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:32:57.786866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:32:57.929967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:32:57.972909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7149e3b9-8ab8-4623-80ed-30aa37906f3b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:32:58.035156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b9cb-e0f9-4ffd-bf13-e60bcac959cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:32:58.036235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b9cb-e0f9-4ffd-bf13-e60bcac959cd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:58.036712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b9cb-e0f9-4ffd-bf13-e60bcac959cd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:58.037175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b9cb-e0f9-4ffd-bf13-e60bcac959cd None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:58.038282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179b9cb-e0f9-4ffd-bf13-e60bcac959cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:32:58.040844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179b9cb-e0f9-4ffd-bf13-e60bcac959cd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:32:58.059776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 495/996] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:32:57 2022] GET /identity/v3/services => generated 1727 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:32:59.630524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f4e99-bd05-4dce-a1b0-544caf408825 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:59.631301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f4e99-bd05-4dce-a1b0-544caf408825 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:59.631445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f4e99-bd05-4dce-a1b0-544caf408825 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:59.634154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 502/997] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:32:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:32:59.642779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:32:59.643538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:32:59.643903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:32:59.674733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:32:59.676175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:32:59.794931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 496/998] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:32:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:00.062496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f4e99-bd05-4dce-a1b0-544caf408825 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:00.214238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f4e99-bd05-4dce-a1b0-544caf408825 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:00.273903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:00.274963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.275431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.275958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] PATH_INFO: `/v3/services/image` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.277098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:00.279732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-08a00098-5d51-4d44-9f25-2553bddeb387 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:00.296882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:00.297971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:00.298831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:00.299573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 503/999] 10.222.0.52 () {56 vars in 1105 bytes} [Fri Oct 28 09:33:00 2022] GET /identity/v3/services/image => generated 86 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:00.305706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:00.377904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853d41-9b84-45d6-a6a5-1725e97fdc65 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.378291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853d41-9b84-45d6-a6a5-1725e97fdc65 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.378737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec853d41-9b84-45d6-a6a5-1725e97fdc65 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.381259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 504/1000] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:00.388896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004a44-b1ad-426e-a445-9930326dd633 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.389368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004a44-b1ad-426e-a445-9930326dd633 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.390011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01004a44-b1ad-426e-a445-9930326dd633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.394832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:00.422498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-01004a44-b1ad-426e-a445-9930326dd633 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:00.424789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-01004a44-b1ad-426e-a445-9930326dd633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:00.435359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52191bb-e763-4b8a-b6ab-0c75d67fd70d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:00.474884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:00.475118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:00.475651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.475954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.476225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.476863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:00.477161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:00.478596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:00.488613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 497/1001] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:00.489750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:00.543339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 505/1002] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:00.607888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876811d2-7185-4cbb-8313-b580e403f5fa None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.608386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876811d2-7185-4cbb-8313-b580e403f5fa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.608853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876811d2-7185-4cbb-8313-b580e403f5fa None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.611469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 506/1003] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:00.620971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.621463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.621894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.647643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:00.649685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:00.658314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb9218-0e3c-4ad1-86e7-ba2372ada7a1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:00.718053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:00.719205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.719695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.720188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.721328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:00.721861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:00.724567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:00.727082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:00.739791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 498/1004] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:00.781464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 507/1005] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:00.813084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:00.933806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39bb15-9a6d-4e99-91e3-6e76bd2a4d1a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:00.978599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:00.980265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:00.980721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:00.981178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:00.982181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:00.984653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:01.001225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:01.002286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:01.004143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 499/1006] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:33:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:01.009885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:01.179515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:01.195532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3747fd-58f3-432b-a0b0-a48cd1e7938b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:01.256159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:01.257224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:01.257688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:01.258206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:01.259359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:01.259902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:01.262755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:01.277683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 508/1007] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:33:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:01.278986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:01.307678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:01.372644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:01.424659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0fcb5-54cb-44ff-9b06-cefe3100a007 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:01.468027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:01.468732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:01.469134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:01.469525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:01.470292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:01.470721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:01.472882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:01.479488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 500/1008] 10.222.0.52 () {56 vars in 1112 bytes} [Fri Oct 28 09:33:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:01.486211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:01.491538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4344ffc-479c-4135-a070-769b9eb30a76 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:01.533434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:01.534387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:01.534833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:01.535262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:01.536341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:01.537996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:01.548635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:01.549805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 28 09:33:01.550751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:01.552940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 509/1009] 10.222.0.52 () {56 vars in 1126 bytes} [Fri Oct 28 09:33:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:01.562580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:01.605902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:01.681036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3eff-e82f-457c-b569-76b497012747 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:01.719070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d30a7-e256-4865-b08c-d3ea2eee7ee4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:01.732302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:01.748086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:01.749106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:01.749513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:01.750493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:01.751294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:01.753030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:01.778702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:01.779716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:01.780254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:01.780739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:01.781896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:01.791840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:01.792095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 501/1010] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:33:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:33:01.792267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:01.800590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 510/1011] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:33:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:01.811203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:02.011426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8a3c9-0354-48ac-9deb-cdc26fea977c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:02.072258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:02.073992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:02.074347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:02.074832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:02.076067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:02.076615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:02.079297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8208f5-d6e1-40d9-a8db-754133f0127c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:02.089857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 502/1012] 10.222.0.52 () {56 vars in 1122 bytes} [Fri Oct 28 09:33:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:02.096429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:02.243397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:02.268810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb75e5-f9c6-4c03-afa2-29906e7acaae None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:02.310890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc9bcc-7c57-4f23-9223-0fcc357c5aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:02.311525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc9bcc-7c57-4f23-9223-0fcc357c5aa0 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:02.311832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc9bcc-7c57-4f23-9223-0fcc357c5aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:02.312103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc9bcc-7c57-4f23-9223-0fcc357c5aa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:02.312810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc9bcc-7c57-4f23-9223-0fcc357c5aa0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:02.314283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc9bcc-7c57-4f23-9223-0fcc357c5aa0 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:02.334847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 511/1013] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:33:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:33:03.720282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c9d03-d646-44b2-83fd-11036c496937 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:03.721076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c9d03-d646-44b2-83fd-11036c496937 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:03.721254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c9d03-d646-44b2-83fd-11036c496937 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:03.724486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 503/1014] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:03.734781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f03fb-f896-47f7-9a6a-9d7323f188ab None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:03.735563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f03fb-f896-47f7-9a6a-9d7323f188ab None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:03.735705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f03fb-f896-47f7-9a6a-9d7323f188ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:03.766112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5b6f03fb-f896-47f7-9a6a-9d7323f188ab None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:03.767653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5b6f03fb-f896-47f7-9a6a-9d7323f188ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:03.852372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 512/1015] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:03 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:04.122084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c9d03-d646-44b2-83fd-11036c496937 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:04.203350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:04.307898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d7de-dc65-41bc-9f7d-c6eb86b61bcf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:04.308157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d7de-dc65-41bc-9f7d-c6eb86b61bcf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:04.308468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514d7de-dc65-41bc-9f7d-c6eb86b61bcf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:04.310229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 513/1016] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:33:04.313700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c9d03-d646-44b2-83fd-11036c496937 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:04.316611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3c94c-d8d3-4d72-afe8-f0d4c2ed4a4b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:04.317051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3c94c-d8d3-4d72-afe8-f0d4c2ed4a4b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:04.317516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3c94c-d8d3-4d72-afe8-f0d4c2ed4a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:04.344665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cca3c94c-d8d3-4d72-afe8-f0d4c2ed4a4b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:04.346506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cca3c94c-d8d3-4d72-afe8-f0d4c2ed4a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:04.381529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a970ac0e-1126-4dd0-8b0a-99acf1e5188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:04.382383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970ac0e-1126-4dd0-8b0a-99acf1e5188a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:04.382821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970ac0e-1126-4dd0-8b0a-99acf1e5188a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:04.383253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970ac0e-1126-4dd0-8b0a-99acf1e5188a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:04.385166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970ac0e-1126-4dd0-8b0a-99acf1e5188a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:04.387548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a970ac0e-1126-4dd0-8b0a-99acf1e5188a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:04.409098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 504/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:33:04 2022] POST /identity/v3/services => generated 242 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:33:04.472360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:04.501063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 514/1018] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:04.774811 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d166-fbba-4e6d-b40e-a21e7acf6e30 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:04.776145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d166-fbba-4e6d-b40e-a21e7acf6e30 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:04.776145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f13d166-fbba-4e6d-b40e-a21e7acf6e30 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:04.777479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 505/1019] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:33:04.783157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:04.783959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:04.783959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:04.804219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:04.805412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:04.817253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13d166-fbba-4e6d-b40e-a21e7acf6e30 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:04.886253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 515/1020] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:04.953120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f13d166-fbba-4e6d-b40e-a21e7acf6e30 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:04.956600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:04.998072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:04.999073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:04.999533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:04.999971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.012300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:05.017459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:05.018480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:33:05.019579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:05.020818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 506/1021] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Oct 28 09:33:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:05.026819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:05.153679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a592d-de73-4ccb-a652-9e02210827bd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:05.189350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:05.201939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:05.202571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:05.202831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:05.203172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.203836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:05.204136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:05.205578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:05.221099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 516/1022] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:33:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:05.221385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:05.222413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:05.351208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ff8a4-bb82-48c1-9f07-64bf340088df None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:05.381863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c655-9775-4913-b947-30d1f208bc7c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:05.390308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:05.391288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:05.391774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:05.392229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.393579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:05.395458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:05.397163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 507/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:05.406182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:05.430001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:05.430657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:05.430934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:05.431207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.443832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:05.445307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:05.460355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:05.462301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:05.464086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:05.465879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:05.465879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:05.465879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:05.465879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 517/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:05.470248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:05.523942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:05.593946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa622b-6c22-4608-8b30-cdee37d2e855 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:05.637832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f2a8-faa8-48ff-9b09-020232950680 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:05.651110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:05.652227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:05.652620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:05.653084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.654227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:05.656974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:05.700222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:05.700951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:05.701278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:05.701540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.702152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:05.702454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:05.703847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:05.707067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 508/1025] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:33:05 2022] POST /identity/v3/projects => generated 311 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:33:05.723820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 518/1026] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:05.730163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:05.837092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:05.876482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fdd955-c5f6-48c5-a090-5f60647c1898 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:05.944658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:05.944658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:05.944658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:05.944658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:05.956242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:05.957728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:33:05.975657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:05.977613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:05.980554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 509/1027] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:33:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:05.985848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:06.156510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:06.162222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbe45-f8e1-4c92-9b9d-4794f1f05771 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:06.220991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:06.221509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:06.221905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:06.222308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:06.223504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:06.224064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:06.226067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:06.259792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 519/1028] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:33:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:06.263568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:06.392815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90776e96-77e4-45fb-809d-606ef80c42ff None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:06.433072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce641bee-0771-4a4d-97dc-cfe49473a7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:06.433684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce641bee-0771-4a4d-97dc-cfe49473a7ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:06.433959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce641bee-0771-4a4d-97dc-cfe49473a7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:06.434228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce641bee-0771-4a4d-97dc-cfe49473a7ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:06.447897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:06.519451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce641bee-0771-4a4d-97dc-cfe49473a7ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:06.521501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce641bee-0771-4a4d-97dc-cfe49473a7ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:06.714151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 510/1029] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:06.723817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:06.918875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35141-3b88-4520-902b-ec88364f9dc3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:06.919615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35141-3b88-4520-902b-ec88364f9dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:06.919615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a35141-3b88-4520-902b-ec88364f9dc3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:06.920889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 511/1030] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:33:06.921438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52ada9-6d5f-463c-827b-17c06820ebce None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:06.926236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:06.926565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:06.926788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:06.946696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:06.948511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:06.989756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:06.993792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:06.995431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:06.995962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:06.996430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:07.016789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:07.019376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:07.041179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 512/1031] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:07.047266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:07.049270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:07.051521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:07.051521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:07.051521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:07.056345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 520/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:07.059612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:07.227108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3350e16-ed4a-4d72-90d8-50fc8c294cdc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:07.269968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:07.271030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:07.271520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:07.272036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:07.272913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:07.273449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:07.275112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:07.290512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 513/1033] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:07.299939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:07.325179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:07.496304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d362c-533a-432d-a0e6-8dd791ec1434 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:07.543397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:07.562836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:07.564012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:07.564505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:07.564979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:07.583731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:07.586941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:07.606773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:33:07.612177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:07.613174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:33:07.614075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:07.614541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 521/1034] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:07.624299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:07.725113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0460e1-3bef-48e8-b7eb-8d9fd4e9d04d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:07.749239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:07.787406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:07.788634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:07.789095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:07.789546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:07.790603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:07.793025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:07.805488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02616dbe-0da5-4c88-a41d-19c2b3c84954 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:07.810624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:07.811677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:07.816842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 514/1035] 10.222.0.52 () {56 vars in 1105 bytes} [Fri Oct 28 09:33:07 2022] GET /identity/v3/services/image => generated 86 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:07.818737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:07.872287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:07.873955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:07.874889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:07.875726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:07.877068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:07.877613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:07.880478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:07.912334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 522/1036] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:33:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:07.919874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:07.934275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:07.942817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ce388-d120-45e8-83c5-466a89e45526 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:07.990995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:07.991647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:07.991949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:07.992200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:07.992857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] RBAC: Adding query filter params (name=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:07.993231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:07.995263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:08.002981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 515/1037] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:08.008687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:08.124765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cadd97-01f6-473e-8f7b-7a0a8e109a53 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:08.170641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb31047-57ed-4524-9653-fb08de188bca None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:08.186293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:08.194787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:08.195862 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:08.196299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:08.196761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:08.198305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] RBAC: Adding query filter params (role.id=9f80f81454c74975ad8943f88fe0e487, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=9dc2bb9976f949548bc4743eb297b082) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:08.198846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:08.201175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e5aba-40c0-45ca-a1db-b83f13585375 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:08.207667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:08.208430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:08.208986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:08.209369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:08.210096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:08.210293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 523/1038] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:07 2022] GET /identity/v3/role_assignments?user.id=9dc2bb9976f949548bc4743eb297b082&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9f80f81454c74975ad8943f88fe0e487 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:08.210804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:08.212924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895fb43-9701-456b-b8e5-84bfc77d6eee None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:08.213219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895fb43-9701-456b-b8e5-84bfc77d6eee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:08.213354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:08.213489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895fb43-9701-456b-b8e5-84bfc77d6eee None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:08.216165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 524/1039] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:08.224803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:08.225238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:08.225675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:08.226597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 516/1040] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:08.233597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:08.255223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:08.257145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:08.278723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:08.387929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 525/1041] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:08.396165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf19902-0012-4a6d-a5f0-496e558aa94d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:08.437736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:08.438366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:08.438641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:08.438964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:08.439687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] RBAC: Adding query filter params (interface=public, service_id=88a81a6aca89466180b916f684fac2d0, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:08.440042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:08.441456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:08.447892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 517/1042] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:33:08 2022] GET /identity/v3/endpoints?service_id=88a81a6aca89466180b916f684fac2d0&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:08.454272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:08.615394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e5057-392c-488c-8089-d98a3018ed53 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:08.657467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:08.671460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:08.672454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:08.672892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:08.673314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:08.674094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:08.675646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:08.683655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 526/1043] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:33:08 2022] GET /identity/v3/services => generated 1957 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:33:08.880228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:09.044000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:09.063710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e046-e70f-4621-ae5c-9e2c09c59fc9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:09.105730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42196ef-945d-4d1d-b1ba-42e657264940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:09.106478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42196ef-945d-4d1d-b1ba-42e657264940 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:09.106780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42196ef-945d-4d1d-b1ba-42e657264940 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:09.107037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42196ef-945d-4d1d-b1ba-42e657264940 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:09.108337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42196ef-945d-4d1d-b1ba-42e657264940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:09.109836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42196ef-945d-4d1d-b1ba-42e657264940 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:09.111925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 518/1044] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:09.118372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:09.280509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3df3ce-6448-4848-85ae-c3cc31a05e79 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:09.319826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da18ff8-1f2c-48ed-8793-c760b9596bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:09.320445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da18ff8-1f2c-48ed-8793-c760b9596bd8 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:09.320707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da18ff8-1f2c-48ed-8793-c760b9596bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:09.320970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da18ff8-1f2c-48ed-8793-c760b9596bd8 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:09.321629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da18ff8-1f2c-48ed-8793-c760b9596bd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:09.323092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da18ff8-1f2c-48ed-8793-c760b9596bd8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:09.359780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 527/1045] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:33:09 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:33:11.333695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b20a50-ac43-4002-8e82-d5c13dbcca7c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:11.334250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b20a50-ac43-4002-8e82-d5c13dbcca7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:11.334250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b20a50-ac43-4002-8e82-d5c13dbcca7c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:11.336117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 519/1046] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:11.343547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d31e4f-1132-4710-b362-f507c5618595 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:11.344398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d31e4f-1132-4710-b362-f507c5618595 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:11.344398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d31e4f-1132-4710-b362-f507c5618595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:11.367110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6d31e4f-1132-4710-b362-f507c5618595 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:11.368322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6d31e4f-1132-4710-b362-f507c5618595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:11.416685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:11.467799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 528/1047] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:11.612441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd17c1-9476-4ec1-9586-d93e405da068 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:11.612758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd17c1-9476-4ec1-9586-d93e405da068 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:11.612967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd17c1-9476-4ec1-9586-d93e405da068 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:11.614461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 520/1048] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:33:11.621801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:11.622347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:11.622833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:11.645423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:11.646635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:11.724639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 529/1049] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:11.885335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd17c1-9476-4ec1-9586-d93e405da068 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.072319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd17c1-9476-4ec1-9586-d93e405da068 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.137049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299657b6-270a-4ff9-94cb-5a6f4df9fd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.138158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299657b6-270a-4ff9-94cb-5a6f4df9fd57 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.138469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299657b6-270a-4ff9-94cb-5a6f4df9fd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.138707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299657b6-270a-4ff9-94cb-5a6f4df9fd57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.150939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299657b6-270a-4ff9-94cb-5a6f4df9fd57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-299657b6-270a-4ff9-94cb-5a6f4df9fd57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:12.155872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:12.157768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:33:12.159551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:12.162635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 521/1050] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Oct 28 09:33:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:12.164336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.203797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc06562-a3d3-43fc-a03b-eb9c22a40ca5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.204315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc06562-a3d3-43fc-a03b-eb9c22a40ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.204808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc06562-a3d3-43fc-a03b-eb9c22a40ca5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.208296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 522/1051] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:12.215528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.215908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.216182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.246235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:12.248380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:12.268568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:12.286404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff05503-0953-4f28-843f-409e9a60b9aa None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.289154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:12.325583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.326086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.326362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.326630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.327306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:12.327643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:12.329024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:12.338907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 530/1052] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:12.339784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.348663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 523/1053] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:12.349450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.487845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66aef9-7786-463f-b9ae-72386848216d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.494820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7874d-032b-4afe-9de0-1d8603f0ef63 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.531265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.531946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.532219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.532486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.533453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:12.534543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:12.542414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.543487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.544011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.544500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:12.544877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:12.545889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:12.546766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 531/1054] 10.222.0.52 () {56 vars in 1105 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/services/image => generated 86 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:12.552678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.567242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:12.569744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:12.582018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:12.592266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:12.595624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:12.596790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:12.597706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:12.597706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:12.597706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:12.597706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 524/1055] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:12.603826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.708083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55a72f-39f3-4592-93ae-2d568ce3d0d4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.739839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51a0bb-2ba5-4d7a-bb0e-4f13721f514d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.776087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.776710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.776978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.777241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.777887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:12.778186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:12.778572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.779486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.779597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:12.780029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.780515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.781816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] RBAC: Adding query filter params (name=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:12.782522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:12.785198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:12.792746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 525/1056] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:12.798826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 532/1057] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:12.799037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.800626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:12.860510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:12.906196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7eab-bc5e-48ed-85ad-0f1a25a1d86f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.943897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:12.944531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:12.944803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:12.945069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:12.955362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:12.955777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:12.957604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:12.967299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b17e5-3bd8-45fa-ac5f-9a686defe9ba None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:33:12.971554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:12.972764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 526/1058] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:12.973754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.015105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.016037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.016438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.016869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.017811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] RBAC: Adding query filter params (type=image) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:13.018177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.019772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.025876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 533/1059] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:13.026720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.113329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598d156-fc25-4aa0-b2dd-3e4da096bb44 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.136502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:13.145770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d59e5-f8b4-4d09-962c-6fe52b205372 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.162183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.162829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.163339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.163851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.182115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:13.183009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.183631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.183933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.184203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.184937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:13.185237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.186647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:13.189819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:13.191711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 527/1060] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:13.194533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.209520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 534/1061] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:13.209951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.232558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:13.320317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15d112d-860b-4e43-9873-9bd168a43eab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.325366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c514d1f-d31f-4cdd-86a0-151bfd0c844f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.356813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.357417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.357691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.357961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.358637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.360072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.363047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.363718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.363973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.364244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.364909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:13.365209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.366573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.376354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 535/1062] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:13.382181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.385593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 528/1063] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:33:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:33:13.386910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.437666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:13.500791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:13.542721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957564c8-016a-4add-8d37-3ea6985091d1 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.569952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308a53b-4012-4de9-898f-f4e9f72d2e01 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.594129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.594785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.595065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.595364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.616931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.620197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.634829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.636118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.636412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.636682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] PATH_INFO: `/v3/users/57ff2d7591804a298205f8f739792fd2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:13.646353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:13.648331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:13.650063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:13.650063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:13.650063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:13.650063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 536/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:13.654493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] RBAC: Authorizing `identity:get_user(user_id=57ff2d7591804a298205f8f739792fd2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.656058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.657807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 529/1065] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/users/57ff2d7591804a298205f8f739792fd2 => generated 285 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:13.660050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.667557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.815481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2c35f-71b7-4926-9a49-e8dc54121a51 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.827487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:13.845207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9a7ea-553d-48cb-8ba7-efedfa23f54a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:13.860850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:13.872454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.873390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.873881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.874361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.875459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:13.876201 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.878689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:13.889116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:13.889933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:13.890341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:13.890738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:13.903479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 537/1066] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:13.911839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:13.990173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:13.991681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:14.048038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746526e1-9d80-4b05-b4c6-2e31d2e9ac27 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:14.093471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:14.094577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:14.094577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:14.094977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:14.102579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:14.104676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:33:14.116620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:14.117780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:14.118763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 538/1067] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:14.121214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:14.127349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:14.144988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 530/1068] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 28 09:33:14.148697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:14.209552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:14.277486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce125405-9541-46e4-93a0-662ef02d9d22 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:14.286726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b66db-d397-4958-9b60-64eaeef7aac4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:14.318447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:14.319099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:14.319578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:14.319853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:14.320864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] RBAC: Authorizing `identity:get_project(project_id=d054c7e25b0949859318815959498163)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:14.322815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:14.324774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 531/1069] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:14 2022] GET /identity/v3/projects/d054c7e25b0949859318815959498163 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:14.330795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:14.352365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:14.353450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:14.354005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:14.354475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:14.356069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:14.356718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:14.359855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:14.387862 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 539/1070] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:14.389765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:14.411851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:14.480299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bc1eb-c78d-48e5-9486-7c4c04517437 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:14.524155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:14.525217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:14.525538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:14.525972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:14.527347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=d054c7e25b0949859318815959498163, user.id=57ff2d7591804a298205f8f739792fd2) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:14.527575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:14.529565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa7e93-75ba-449a-9243-de560c7ce4a5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:14.535819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704be7e-370d-4c8a-9ab6-28c2fe1f56bc None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:14.537537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 532/1071] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:14 2022] GET /identity/v3/role_assignments?user.id=57ff2d7591804a298205f8f739792fd2&scope.project.id=d054c7e25b0949859318815959498163&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:14.551850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:14.585967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927206b1-5d01-4a1c-b608-15fa095eed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:14.586665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927206b1-5d01-4a1c-b608-15fa095eed2b None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:14.586933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927206b1-5d01-4a1c-b608-15fa095eed2b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:14.587197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927206b1-5d01-4a1c-b608-15fa095eed2b None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/9dc2bb9976f949548bc4743eb297b082/roles/9f80f81454c74975ad8943f88fe0e487` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:14.589664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927206b1-5d01-4a1c-b608-15fa095eed2b None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=9dc2bb9976f949548bc4743eb297b082, role_id=9f80f81454c74975ad8943f88fe0e487)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:14.591129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927206b1-5d01-4a1c-b608-15fa095eed2b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:14.608397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 540/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:33:14 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/9dc2bb9976f949548bc4743eb297b082/roles/9f80f81454c74975ad8943f88fe0e487 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:33:15.854930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235682e1-168f-4402-a40a-9e2a3e0234a1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:15.855676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235682e1-168f-4402-a40a-9e2a3e0234a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:15.855676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235682e1-168f-4402-a40a-9e2a3e0234a1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:15.856868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 533/1073] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:15.864107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:15.864853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:15.865340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:15.894547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:15.896911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:15.991789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 541/1074] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:16.333563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235682e1-168f-4402-a40a-9e2a3e0234a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:16.445705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235682e1-168f-4402-a40a-9e2a3e0234a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:16.469592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:16.487687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad408ad-c047-4729-9a98-77c6b058f755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:16.488332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad408ad-c047-4729-9a98-77c6b058f755 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:16.488625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad408ad-c047-4729-9a98-77c6b058f755 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:16.488940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad408ad-c047-4729-9a98-77c6b058f755 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:16.568209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad408ad-c047-4729-9a98-77c6b058f755 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:16.569748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad408ad-c047-4729-9a98-77c6b058f755 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:16.695909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 534/1075] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 28 09:33:16.697363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:16.815917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0d60a-99d8-4828-bb12-fb49a2fc7b5b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:16.816280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0d60a-99d8-4828-bb12-fb49a2fc7b5b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:16.816468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0d60a-99d8-4828-bb12-fb49a2fc7b5b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:16.818017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 535/1076] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:16.823309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:16.823583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:16.823890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:16.830254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:16.836504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a1150-35e2-4a0b-883d-085ea7893b5e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:16.844700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:16.845904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:16.863119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:16.889505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:16.890206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:16.890425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:16.890694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:16.902324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:16.903840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:16.921136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:16.922173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:16.923183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:16.923183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:16.923183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:16.927414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 542/1077] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:16.928619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:16.955913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 536/1078] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:17.054423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd708482-a392-4513-aec0-a41a0194b561 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:17.095090 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:17.095710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.096041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.096292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.097003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:17.097307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:17.098687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:17.116017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 543/1079] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:17.228129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:17.362290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e337fa3-ae81-4ec4-8ce1-f80671502bb5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:17.405065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:17.406130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.406610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.407084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.423413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:17.428897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:17.430599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:17.432775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 537/1080] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:17.438153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:17.456960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:17.575966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a2e5f-5140-44b7-be6d-1e2b784ca382 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:17.617042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:17.617670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.617945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.618213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.618952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:17.619289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:17.620747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:17.630288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 544/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:17.636587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:17.652110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:17.749834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320ab1c-8516-4e80-b26f-8481cc2c2e24 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:17.787258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7677e3f-109b-4c34-b2d5-f7d90a6141cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:17.788029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7677e3f-109b-4c34-b2d5-f7d90a6141cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.788309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7677e3f-109b-4c34-b2d5-f7d90a6141cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.788582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7677e3f-109b-4c34-b2d5-f7d90a6141cc None admin] PATH_INFO: `/v3/users/57ff2d7591804a298205f8f739792fd2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.789767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7677e3f-109b-4c34-b2d5-f7d90a6141cc None admin] RBAC: Authorizing `identity:get_user(user_id=57ff2d7591804a298205f8f739792fd2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:17.791815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7677e3f-109b-4c34-b2d5-f7d90a6141cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:17.793459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 538/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:17 2022] GET /identity/v3/users/57ff2d7591804a298205f8f739792fd2 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:17.799323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:17.813213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc692a-8dd1-4e04-8cb6-e9a1265b2846 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.813701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc692a-8dd1-4e04-8cb6-e9a1265b2846 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.814174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc692a-8dd1-4e04-8cb6-e9a1265b2846 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.816873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 539/1083] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:17.822988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.823270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.823545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.843218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:17.844376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:17.913286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:17.937136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 540/1084] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:17.950572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833912c-e5de-496b-86db-90edeae22a34 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:17.994351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:17.995157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:17.995533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:17.995970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:17.997565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] RBAC: Authorizing `identity:get_project(project_id=d054c7e25b0949859318815959498163)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:17.999240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:18.001368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 545/1085] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:17 2022] GET /identity/v3/projects/d054c7e25b0949859318815959498163 => generated 311 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:18.008234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:18.022522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:18.137699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1910bd-07e2-48bb-8f58-f51cabc3bb0b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:18.177329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:18.178282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:18.178557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:18.178815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163/users/57ff2d7591804a298205f8f739792fd2/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:18.180899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] RBAC: Authorizing `identity:create_grant(project_id=d054c7e25b0949859318815959498163, user_id=57ff2d7591804a298205f8f739792fd2, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:18.183132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:18.201612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 541/1086] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:33:18 2022] PUT /identity/v3/projects/d054c7e25b0949859318815959498163/users/57ff2d7591804a298205f8f739792fd2/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:33:18.230555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:18.418697 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e194-01ad-4a13-8be6-134033e16c44 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:18.478244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:18.480131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:18.480611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:18.481091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:18.499936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:18.509411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:18.511716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:18.513670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 546/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Fri Oct 28 09:33:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:18.521557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:18.638039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:18.683216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab8ff4-b14c-4801-9415-f0321af563ff None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:18.721465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:18.722204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:18.722579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:18.722939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:18.723710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:18.724142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:18.725636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:18.736088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 542/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:33:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:18.742426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:18.746857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:18.863463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780586-e707-43c1-aa50-e582a6f8c850 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:18.902991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:18.903732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:18.904121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:18.904459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:18.917976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:18.919477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:18.936630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:18.938866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:18.940988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:18.940988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:18.940988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:18.942696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 547/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:18.943867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:19.064417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963ceff-1b26-4518-b218-f56a6d7950cb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:19.100380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:19.108700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:19.109409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:19.109749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:19.110079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:19.110930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:19.111294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:19.112904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:19.127732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 543/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:19.137876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:19.209486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:19.267318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1516c-7ecf-45ca-9376-142cde16e2a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:19.315146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:19.316222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:19.316697 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:19.317151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:19.332291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:19.335115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:33:19.358420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:19.360466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:19.362449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 548/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Fri Oct 28 09:33:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:19.368714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:19.504299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28164a-0876-4daf-99c2-22545c221e47 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:19.571126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:19.571652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:19.572719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:19.573203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:19.574581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:19.575126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:19.578094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:19.612537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 544/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Fri Oct 28 09:33:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:19.619562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:19.737592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df1c07-245f-4e0f-91db-e3067805e3a9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:19.752569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:19.775129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:19.775663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:19.775963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:19.776230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:19.844384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:19.845746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:19.945502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 549/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:19.952279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:20.004097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:20.059789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11da0d9-c0a3-441b-b6d3-82a6baca19e8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:20.095571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:20.096228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.096501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.096768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.108071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:20.109526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:20.125424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:20.127407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:33:20.128606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:20.128606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:33:20.128606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:20.132816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 545/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:20.133243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:20.150300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:20.243422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aaf0fd-fa5a-4af2-b5b6-1f6c1f2abf27 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:20.279495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:20.280127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.280399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.280664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.281292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:20.281583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:20.282938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:20.295082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 550/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:20.301010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:20.409452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:20.412693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab7e63-16ef-4184-b240-d1a8979855c1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:20.453094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:20.453726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.453997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.454262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.461609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:20.463615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-aa337cd2-c444-45ab-9882-0ca4e0195e61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:33:20.475059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:20.476867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:20.479255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 546/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:20.484268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:20.529040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a9c5d-7c5b-4959-b2f9-4d8e48fa940b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.529309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a9c5d-7c5b-4959-b2f9-4d8e48fa940b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.529575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a9c5d-7c5b-4959-b2f9-4d8e48fa940b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.530931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 547/1097] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:33:20.536783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.537047 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.537326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.558719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:20.559994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:20.569971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:20.600778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583647d8-4be5-48a7-b021-bae3727836e7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:20.639576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:20.640226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.640500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.640768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.641469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:20.641770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:20.642675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 548/1098] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:20.643309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:20.657582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 551/1099] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:20.664228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:20.765177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:20.776450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbeb6d-7d2b-42d6-8c31-ba5507b4c216 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:20.810672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:20.811277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:20.811552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:20.811848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:20.812568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] RBAC: Adding query filter params (role.id=9f80f81454c74975ad8943f88fe0e487, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=9dc2bb9976f949548bc4743eb297b082) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:20.812872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:20.814311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:20.820414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 549/1100] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:20 2022] GET /identity/v3/role_assignments?user.id=9dc2bb9976f949548bc4743eb297b082&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9f80f81454c74975ad8943f88fe0e487 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:20.919471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:21.047869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34bfc3-b4f4-49a8-a11b-b5b589c1b09b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:21.096033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:21.096835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:21.097207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:21.097569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:21.100087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:21.114453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:21.121259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:21.123199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:21.124788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 552/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:33:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:21.131902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:21.274496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba866a3-b403-49c9-8d22-c43af7a416e0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:21.328321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:21.329374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:21.329859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:21.330304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:21.331487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:21.332070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:21.334686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:21.351908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 550/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:33:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:21.359858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:21.492961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a28c3a-7222-4916-8d4c-da7e9ee80596 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:21.544123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:21.545043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:21.545393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:21.545800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] PATH_INFO: `/v3/users/57ff2d7591804a298205f8f739792fd2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:21.547287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] RBAC: Authorizing `identity:get_user(user_id=57ff2d7591804a298205f8f739792fd2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:21.549309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:21.551117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 553/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:21 2022] GET /identity/v3/users/57ff2d7591804a298205f8f739792fd2 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:21.560869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:21.587670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:21.689049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed100d77-2b0d-4c1e-9093-1782f879524b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:21.729247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:21.730132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:21.730478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:21.730929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:21.824334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:21.826279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:21.852651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:21.963976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 551/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:21.970137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:22.033575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:22.074934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1aa00d-00de-4a99-9f3f-ea98b0f4e5da None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:22.110708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63628a4-835c-4371-8656-068f58121296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:22.111338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63628a4-835c-4371-8656-068f58121296 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:22.111614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63628a4-835c-4371-8656-068f58121296 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:22.111915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63628a4-835c-4371-8656-068f58121296 None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:22.113048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63628a4-835c-4371-8656-068f58121296 None admin] RBAC: Authorizing `identity:get_project(project_id=d054c7e25b0949859318815959498163)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:22.114479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63628a4-835c-4371-8656-068f58121296 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:22.116171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 554/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:21 2022] GET /identity/v3/projects/d054c7e25b0949859318815959498163 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:22.122521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:22.244888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b8a96-7518-4b32-8990-3fb309e38554 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:22.278118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:22.311302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:22.312386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:22.312877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:22.313357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:22.314802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=d054c7e25b0949859318815959498163, user.id=57ff2d7591804a298205f8f739792fd2) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:22.315333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:22.318203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf3c65-6ce4-47ed-9a5a-ff9112bac111 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:22.330265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 552/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:22 2022] GET /identity/v3/role_assignments?user.id=57ff2d7591804a298205f8f739792fd2&scope.project.id=d054c7e25b0949859318815959498163&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:24.597791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f10c0-cd5a-4aba-9a0d-6dee3896885a None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:24.598343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f10c0-cd5a-4aba-9a0d-6dee3896885a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:24.598343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f10c0-cd5a-4aba-9a0d-6dee3896885a None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:24.599698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 555/1107] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:24.606945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:24.607459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:24.607901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:24.637288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:24.639260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:24.752633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 553/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:25.025960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f10c0-cd5a-4aba-9a0d-6dee3896885a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:25.141974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f10c0-cd5a-4aba-9a0d-6dee3896885a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:25.174386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:25.184846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f73deb-a8f4-40bb-817a-cfe62ff06027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:25.185493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f73deb-a8f4-40bb-817a-cfe62ff06027 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:25.185770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f73deb-a8f4-40bb-817a-cfe62ff06027 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:25.186040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f73deb-a8f4-40bb-817a-cfe62ff06027 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:25.187188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f73deb-a8f4-40bb-817a-cfe62ff06027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:25.188672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f73deb-a8f4-40bb-817a-cfe62ff06027 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:25.190322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 556/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:25.199144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:25.306990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0448b6-9931-4dc4-91e1-de40a264d8c4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:25.318505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:25.347938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fdcdc-1cd8-401d-9c82-11dea36dc325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:25.348561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fdcdc-1cd8-401d-9c82-11dea36dc325 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:25.348838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fdcdc-1cd8-401d-9c82-11dea36dc325 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:25.349109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fdcdc-1cd8-401d-9c82-11dea36dc325 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:25.349779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fdcdc-1cd8-401d-9c82-11dea36dc325 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:25.351233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fdcdc-1cd8-401d-9c82-11dea36dc325 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:25.373558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 554/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:33:25 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:33:27.653076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971143-9c31-43b8-915c-1a2e6af3060f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:27.653644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971143-9c31-43b8-915c-1a2e6af3060f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:27.653644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1971143-9c31-43b8-915c-1a2e6af3060f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:27.655076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 557/1111] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:33:27.660980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:27.661507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:27.661507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:27.680269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:27.682225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:27.752864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 555/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:28.025882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971143-9c31-43b8-915c-1a2e6af3060f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:28.140797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1971143-9c31-43b8-915c-1a2e6af3060f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:28.179362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:28.180076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:28.180393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:28.180702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] PATH_INFO: `/v3/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:28.182185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] RBAC: Authorizing `identity:get_role(role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:28.183589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:28.185011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 558/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:28 2022] GET /identity/v3/roles/7682e93d1a8e439ba530f0911772aa20 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:28.191204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:28.298868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:28.307025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460136a-d041-45f4-9c01-43fe5076f754 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:28.346277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:28.346981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:28.347303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:28.347615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] PATH_INFO: `/v3/users/bbbb5146158a49aab0aa7fdda5a1a52b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:28.363164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] RBAC: Authorizing `identity:get_user(user_id=bbbb5146158a49aab0aa7fdda5a1a52b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:28.364664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:28.366361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 556/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:28 2022] GET /identity/v3/users/bbbb5146158a49aab0aa7fdda5a1a52b => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:28.372580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:28.441138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:28.481540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edbece-ee49-4b33-9455-45f8f8906a76 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:28.517149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:28.517766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:28.518046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:28.518319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:28.589695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:28.591061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:28.659059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:28.697948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 559/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 28 09:33:28.704072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:28.806988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda34b4-f981-4f2d-9cb1-89c626710d25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:28.842278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c43a24-f31d-41ab-b75e-32d0d7cab740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:28.842833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c43a24-f31d-41ab-b75e-32d0d7cab740 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:28.843108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c43a24-f31d-41ab-b75e-32d0d7cab740 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:28.843379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c43a24-f31d-41ab-b75e-32d0d7cab740 None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:28.844539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c43a24-f31d-41ab-b75e-32d0d7cab740 None admin] RBAC: Authorizing `identity:get_project(project_id=d054c7e25b0949859318815959498163)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:28.845986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c43a24-f31d-41ab-b75e-32d0d7cab740 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:28.847644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 557/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:28 2022] GET /identity/v3/projects/d054c7e25b0949859318815959498163 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:28.857304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:28.965666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea22ee-32aa-4070-84d2-2833224c81f2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:29.000971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:29.001577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:29.001854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:29.002122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:29.002849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] RBAC: Adding query filter params (role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=d054c7e25b0949859318815959498163, user.id=bbbb5146158a49aab0aa7fdda5a1a52b) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:29.003154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:29.004622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779c1701-df54-47fc-8c23-9da9bdd2af64 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:29.010551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 560/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:28 2022] GET /identity/v3/role_assignments?user.id=bbbb5146158a49aab0aa7fdda5a1a52b&scope.project.id=d054c7e25b0949859318815959498163&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:31.937550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814964-ece0-4096-bb76-30c05447f46d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:31.938112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814964-ece0-4096-bb76-30c05447f46d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:31.938511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78814964-ece0-4096-bb76-30c05447f46d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:31.941028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 558/1118] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:31.947322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:31.947799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:31.947924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:31.966652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:31.967864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:31.993138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:32.039277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 561/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:33:32.311138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814964-ece0-4096-bb76-30c05447f46d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:32.339265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:32.422720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78814964-ece0-4096-bb76-30c05447f46d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:32.456829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:32.457465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:32.457745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:32.458018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] PATH_INFO: `/v3/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:32.459427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] RBAC: Authorizing `identity:get_role(role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:32.461143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:32.462980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 559/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:32 2022] GET /identity/v3/roles/7682e93d1a8e439ba530f0911772aa20 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:32.469179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:32.573698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4af2fc-d10e-4ec4-823f-816d7abbd076 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:32.609936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:32.610562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:32.610837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:32.611106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] PATH_INFO: `/v3/users/bbbb5146158a49aab0aa7fdda5a1a52b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:32.612265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=bbbb5146158a49aab0aa7fdda5a1a52b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:32.613688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:32.615298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 562/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:32 2022] GET /identity/v3/users/bbbb5146158a49aab0aa7fdda5a1a52b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:32.623997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:32.720325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:32.731522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6274169-5aa0-4062-92e8-881a366d70ce None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:32.766345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7dc85-9e33-4453-85de-ec3cc272d20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:32.766974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7dc85-9e33-4453-85de-ec3cc272d20e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:32.767248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7dc85-9e33-4453-85de-ec3cc272d20e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:32.767516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7dc85-9e33-4453-85de-ec3cc272d20e None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:32.768669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7dc85-9e33-4453-85de-ec3cc272d20e None admin] RBAC: Authorizing `identity:get_project(project_id=d054c7e25b0949859318815959498163)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:32.770121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7dc85-9e33-4453-85de-ec3cc272d20e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:32.771804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 560/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:32 2022] GET /identity/v3/projects/d054c7e25b0949859318815959498163 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:32.778705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:32.809505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:32.888051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78e6f9-0b8e-4491-9c69-842692872ff4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:32.923074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fbb75-6f93-45d0-ab52-d5e56075e7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:32.923736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fbb75-6f93-45d0-ab52-d5e56075e7dc None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:32.924040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fbb75-6f93-45d0-ab52-d5e56075e7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:32.924310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fbb75-6f93-45d0-ab52-d5e56075e7dc None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163/users/bbbb5146158a49aab0aa7fdda5a1a52b/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:32.926469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fbb75-6f93-45d0-ab52-d5e56075e7dc None admin] RBAC: Authorizing `identity:create_grant(project_id=d054c7e25b0949859318815959498163, user_id=bbbb5146158a49aab0aa7fdda5a1a52b, role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:32.928004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fbb75-6f93-45d0-ab52-d5e56075e7dc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:32.942035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 563/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:33:32 2022] PUT /identity/v3/projects/d054c7e25b0949859318815959498163/users/bbbb5146158a49aab0aa7fdda5a1a52b/roles/7682e93d1a8e439ba530f0911772aa20 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:33:35.152532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ff03b-388c-402d-962b-94b7bcfc08c9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:35.153042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ff03b-388c-402d-962b-94b7bcfc08c9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:35.153042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ff03b-388c-402d-962b-94b7bcfc08c9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:35.154413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 561/1124] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:35.160226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:35.160730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:35.160853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:35.179418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:35.180638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:35.248805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 564/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:35.525198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ff03b-388c-402d-962b-94b7bcfc08c9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:35.628913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ff03b-388c-402d-962b-94b7bcfc08c9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:35.662622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:35.663247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:35.663517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:35.663805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] PATH_INFO: `/v3/roles/7682e93d1a8e439ba530f0911772aa20` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:35.665158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] RBAC: Authorizing `identity:get_role(role_id=7682e93d1a8e439ba530f0911772aa20)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:35.666551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:35.668116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 562/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:35 2022] GET /identity/v3/roles/7682e93d1a8e439ba530f0911772aa20 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:35.674058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:35.692613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:35.775286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a67443-424a-47d3-9b3d-103c444132ea None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:35.812215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:35.812850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:35.813123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:35.813391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] PATH_INFO: `/v3/users/bbbb5146158a49aab0aa7fdda5a1a52b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:35.814506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] RBAC: Authorizing `identity:get_user(user_id=bbbb5146158a49aab0aa7fdda5a1a52b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:35.816058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:35.817543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 565/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:35 2022] GET /identity/v3/users/bbbb5146158a49aab0aa7fdda5a1a52b => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:35.823796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:35.851371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:35.925441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04180cc3-b2bb-4656-91e7-ec66cfd7931e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:35.958780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:35.959381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:35.959648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:35.960016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:36.037299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:36.038700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:36.069558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:36.151032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 563/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:36.157503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:36.249904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:36.265898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812957a-3799-4226-bd0c-6227f1c01f17 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:36.300182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd707a-4cdc-4dd8-a023-7b5588bfd5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:36.300728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd707a-4cdc-4dd8-a023-7b5588bfd5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:36.301005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd707a-4cdc-4dd8-a023-7b5588bfd5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:36.301274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd707a-4cdc-4dd8-a023-7b5588bfd5f6 None admin] PATH_INFO: `/v3/projects/d054c7e25b0949859318815959498163` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:36.302389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd707a-4cdc-4dd8-a023-7b5588bfd5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=d054c7e25b0949859318815959498163)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:36.303826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd707a-4cdc-4dd8-a023-7b5588bfd5f6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:36.305465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 566/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:36 2022] GET /identity/v3/projects/d054c7e25b0949859318815959498163 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:36.311935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:36.407639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545f79f-9fdd-45c0-b519-74be2d7690ea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:36.432867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:36.442146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:36.442745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:36.443019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:36.443286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:36.444028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] RBAC: Adding query filter params (role.id=7682e93d1a8e439ba530f0911772aa20, scope.project.id=d054c7e25b0949859318815959498163, user.id=bbbb5146158a49aab0aa7fdda5a1a52b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:36.444334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:36.445764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad80ca-c870-41b8-a264-a75be62da8c0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:36.451444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 564/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:36 2022] GET /identity/v3/role_assignments?user.id=bbbb5146158a49aab0aa7fdda5a1a52b&scope.project.id=d054c7e25b0949859318815959498163&role.id=7682e93d1a8e439ba530f0911772aa20 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:38.673234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0e5bd-75fd-4ba9-a8a6-40676ba4c208 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:38.673799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0e5bd-75fd-4ba9-a8a6-40676ba4c208 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:38.674003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0e5bd-75fd-4ba9-a8a6-40676ba4c208 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:38.675768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 567/1131] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:38.682153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:38.682631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:38.682722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:38.701048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:38.702225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:38.800417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 565/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:39.074415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0e5bd-75fd-4ba9-a8a6-40676ba4c208 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:39.185230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0e5bd-75fd-4ba9-a8a6-40676ba4c208 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:39.219196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c332324-6d30-420a-a4c8-eca5e475d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:39.219844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c332324-6d30-420a-a4c8-eca5e475d34f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:39.220116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c332324-6d30-420a-a4c8-eca5e475d34f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:39.220382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c332324-6d30-420a-a4c8-eca5e475d34f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:39.221649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c332324-6d30-420a-a4c8-eca5e475d34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:39.222942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c332324-6d30-420a-a4c8-eca5e475d34f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:39.224798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 568/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:39.231142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:39.333876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9d9a0-9397-45e2-a310-5921115ea1ae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:39.356675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:39.372841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bdf119-d825-4ee4-a022-cc9a141a24bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:39.373460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdf119-d825-4ee4-a022-cc9a141a24bf None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:39.373738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdf119-d825-4ee4-a022-cc9a141a24bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:39.374011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bdf119-d825-4ee4-a022-cc9a141a24bf None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:39.374675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdf119-d825-4ee4-a022-cc9a141a24bf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:39.376163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bdf119-d825-4ee4-a022-cc9a141a24bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:39.409414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 566/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:33:39 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:33:41.681820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51c7e7-c46a-442f-ae96-c54251972bf0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:41.682401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51c7e7-c46a-442f-ae96-c54251972bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:41.682401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51c7e7-c46a-442f-ae96-c54251972bf0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:41.683680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 569/1135] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:41.689683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:41.690195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:41.690195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:41.708930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:41.710104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:41.779725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 567/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:42.060861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51c7e7-c46a-442f-ae96-c54251972bf0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:42.084975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:42.170388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51c7e7-c46a-442f-ae96-c54251972bf0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:42.203891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37980bd-63fe-476f-abb3-98b66ee39066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:42.204529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37980bd-63fe-476f-abb3-98b66ee39066 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:42.204807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37980bd-63fe-476f-abb3-98b66ee39066 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:42.205074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37980bd-63fe-476f-abb3-98b66ee39066 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:42.206200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37980bd-63fe-476f-abb3-98b66ee39066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:42.207633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37980bd-63fe-476f-abb3-98b66ee39066 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:42.209275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 570/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:33:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:42.216138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:42.313381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56313c1a-3dbb-4023-b9e1-6a1337bea8e3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:42.314843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:42.350335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd7c449-9e95-4154-973c-8d4e8d87abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:42.350941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7c449-9e95-4154-973c-8d4e8d87abad None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:42.351214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7c449-9e95-4154-973c-8d4e8d87abad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:42.351478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7c449-9e95-4154-973c-8d4e8d87abad None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:42.352157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7c449-9e95-4154-973c-8d4e8d87abad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:42.354203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd7c449-9e95-4154-973c-8d4e8d87abad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:42.376191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 568/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:33:42 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:33:44.619703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64610e5-f0bc-4c7c-978d-943868d2513d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:44.620299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64610e5-f0bc-4c7c-978d-943868d2513d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:44.620299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64610e5-f0bc-4c7c-978d-943868d2513d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:44.621611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 571/1139] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:44.627408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:44.627909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:44.628012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:44.647328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:44.648605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:44.722733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 569/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:44.993790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64610e5-f0bc-4c7c-978d-943868d2513d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:45.090301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:45.101311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64610e5-f0bc-4c7c-978d-943868d2513d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:45.136059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:45.136698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:45.136970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:45.137244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:45.147936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:45.153445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:45.154919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:45.156000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 572/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:33:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:45.163992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:45.249680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:45.263785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75931507-8272-4e85-92f8-ed00aacadca5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:45.296647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:45.297180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:45.297454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:45.297724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:45.298385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:45.298686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:45.300075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:45.308978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 570/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:33:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:45.315157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:45.414889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a963-2e1d-46a5-b602-5cf122923ece None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:45.450109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:45.450741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:45.451020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:45.451292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] PATH_INFO: `/v3/users/429a1599774048999fa4e359bdff5cba` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:45.467128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] RBAC: Authorizing `identity:get_user(user_id=429a1599774048999fa4e359bdff5cba)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:45.468597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:45.470280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 573/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:45 2022] GET /identity/v3/users/429a1599774048999fa4e359bdff5cba => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:45.476383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:45.580328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c5f92-c089-4562-a21b-2571626d309b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:45.624851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:45.625462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:45.625739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:45.626012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:45.644519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:45.694679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:45.696201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:45.795989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 571/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:45.802108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:45.814693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:45.906852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe94f7-b2a2-4db1-a671-f9d567c270a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:45.941783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523703d2-e5d0-4175-8741-a2e922b887ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:45.942416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523703d2-e5d0-4175-8741-a2e922b887ad None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:45.942687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523703d2-e5d0-4175-8741-a2e922b887ad None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:45.942955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523703d2-e5d0-4175-8741-a2e922b887ad None admin] PATH_INFO: `/v3/projects/ffe5e292cbc8434599cd85f872aa3371` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:45.944154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523703d2-e5d0-4175-8741-a2e922b887ad None admin] RBAC: Authorizing `identity:get_project(project_id=ffe5e292cbc8434599cd85f872aa3371)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:45.945567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523703d2-e5d0-4175-8741-a2e922b887ad None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:45.947275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 574/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:45 2022] GET /identity/v3/projects/ffe5e292cbc8434599cd85f872aa3371 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:45.953858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:46.004546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:46.058172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a120f-1386-41ae-8db9-f258a6455a99 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:46.092637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:46.093243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:46.093517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:46.093790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:46.094511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=ffe5e292cbc8434599cd85f872aa3371, user.id=429a1599774048999fa4e359bdff5cba) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:46.094815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:46.096296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef36ade-9e85-4b25-9d0c-0a0485e3cafa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:46.101945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 572/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:45 2022] GET /identity/v3/role_assignments?user.id=429a1599774048999fa4e359bdff5cba&scope.project.id=ffe5e292cbc8434599cd85f872aa3371&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:48.279538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0830-5eb8-4828-9235-aaf638ab9710 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:48.280113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0830-5eb8-4828-9235-aaf638ab9710 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:48.280113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0830-5eb8-4828-9235-aaf638ab9710 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:48.281437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 575/1147] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:48.287444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:48.288054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:48.288152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:48.310590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:48.312142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:48.385146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 573/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:48.650758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e0830-5eb8-4828-9235-aaf638ab9710 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:48.728948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:48.752206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e0830-5eb8-4828-9235-aaf638ab9710 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:48.786394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:48.787019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:48.787289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:48.787559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:48.797537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:48.802252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:48.804160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:48.805989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 576/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:33:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:48.809583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:48.855511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:48.909660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03ebaf-6038-4120-9d89-0b903ab9b36e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:48.943622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:48.944263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:48.944540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:48.944809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:48.945495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:48.945777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:48.947831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:48.956479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 574/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:33:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:48.962846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:49.063685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7b1a8-4c2a-4b46-bae2-79dc787f0770 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:49.097546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:49.098169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:49.098443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:49.098709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] PATH_INFO: `/v3/users/429a1599774048999fa4e359bdff5cba` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:49.099843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] RBAC: Authorizing `identity:get_user(user_id=429a1599774048999fa4e359bdff5cba)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:49.101255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:49.102862 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 577/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:48 2022] GET /identity/v3/users/429a1599774048999fa4e359bdff5cba => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:49.108844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:49.210213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154d905-e000-47d1-a277-bc7e1336dbaa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:49.217398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:49.244057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e528d3-a011-404a-97a3-2e3d0a53b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:49.244682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e528d3-a011-404a-97a3-2e3d0a53b8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:49.244966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e528d3-a011-404a-97a3-2e3d0a53b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:49.245229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e528d3-a011-404a-97a3-2e3d0a53b8a1 None admin] PATH_INFO: `/v3/projects/ffe5e292cbc8434599cd85f872aa3371` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:49.246335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e528d3-a011-404a-97a3-2e3d0a53b8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=ffe5e292cbc8434599cd85f872aa3371)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:49.247786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e528d3-a011-404a-97a3-2e3d0a53b8a1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:49.249405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 575/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:49 2022] GET /identity/v3/projects/ffe5e292cbc8434599cd85f872aa3371 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:49.255562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:49.260013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:49.355491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3d93d-f18a-4c75-9d64-b4f366925579 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:49.388911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87847b-8744-4c6d-b6d3-54702a775ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:49.389570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87847b-8744-4c6d-b6d3-54702a775ede None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:49.389846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87847b-8744-4c6d-b6d3-54702a775ede None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:49.390112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87847b-8744-4c6d-b6d3-54702a775ede None admin] PATH_INFO: `/v3/projects/ffe5e292cbc8434599cd85f872aa3371/users/429a1599774048999fa4e359bdff5cba/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:49.391999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87847b-8744-4c6d-b6d3-54702a775ede None admin] RBAC: Authorizing `identity:create_grant(project_id=ffe5e292cbc8434599cd85f872aa3371, user_id=429a1599774048999fa4e359bdff5cba, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:49.393413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87847b-8744-4c6d-b6d3-54702a775ede None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:49.407160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 578/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:33:49 2022] PUT /identity/v3/projects/ffe5e292cbc8434599cd85f872aa3371/users/429a1599774048999fa4e359bdff5cba/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:33:51.578931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95adcc9-e7a1-47bb-acd1-4cbe7d83f3e1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:51.579511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95adcc9-e7a1-47bb-acd1-4cbe7d83f3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:51.579511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95adcc9-e7a1-47bb-acd1-4cbe7d83f3e1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:51.580830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 576/1154] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:51.586525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:51.587665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:51.587665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:51.607508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:51.608863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:51.678474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:51.688897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 579/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:51.967535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95adcc9-e7a1-47bb-acd1-4cbe7d83f3e1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:52.073889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95adcc9-e7a1-47bb-acd1-4cbe7d83f3e1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:52.107806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:52.108430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:52.108701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:52.108963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:52.119674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:33:52.124486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:52.125575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:33:52.126422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 577/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:33:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:33:52.131960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:52.233604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eaacf-4436-4b24-abf7-1631722e5def None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:52.269005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:52.269622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:52.269897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:52.270170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:52.270835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:52.271135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:52.272534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:52.281477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 580/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:33:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:52.287629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:52.322978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:52.390051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fdfd0-11b9-469d-b703-ffde251990d4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:52.424898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:52.425532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:52.426171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:52.426171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] PATH_INFO: `/v3/users/429a1599774048999fa4e359bdff5cba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:52.427176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] RBAC: Authorizing `identity:get_user(user_id=429a1599774048999fa4e359bdff5cba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:52.428646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:52.430235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 578/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:33:52 2022] GET /identity/v3/users/429a1599774048999fa4e359bdff5cba => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:52.436762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:52.494274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:52.538794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb76b86-4131-4a82-a818-e0a0d8de1444 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:52.574435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:52.575053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:52.575325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:52.575590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:52.649574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:52.651038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:52.711593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:52.757366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 581/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:52.763550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:52.862223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf67d6-86fc-4fa7-810a-4e78ede713b7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:52.863735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:52.897116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640051b-12a1-422d-b766-52d0cf45eaeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:52.897746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640051b-12a1-422d-b766-52d0cf45eaeb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:52.898023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640051b-12a1-422d-b766-52d0cf45eaeb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:52.898294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640051b-12a1-422d-b766-52d0cf45eaeb None admin] PATH_INFO: `/v3/projects/ffe5e292cbc8434599cd85f872aa3371` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:52.899440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640051b-12a1-422d-b766-52d0cf45eaeb None admin] RBAC: Authorizing `identity:get_project(project_id=ffe5e292cbc8434599cd85f872aa3371)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:52.900923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640051b-12a1-422d-b766-52d0cf45eaeb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:52.902552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 579/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:33:52 2022] GET /identity/v3/projects/ffe5e292cbc8434599cd85f872aa3371 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:52.908749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:53.004970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4376fb6-daf9-4f23-9294-d98ed97686b6 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:53.038421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:53.038946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:53.039217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:53.039482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:53.040813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=ffe5e292cbc8434599cd85f872aa3371, user.id=429a1599774048999fa4e359bdff5cba) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:53.041118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:53.042559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b6bc84-ee33-4119-ae33-a461bfff70f6 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:53.048943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 582/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:33:52 2022] GET /identity/v3/role_assignments?user.id=429a1599774048999fa4e359bdff5cba&scope.project.id=ffe5e292cbc8434599cd85f872aa3371&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:33:55.244225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9b26-c7b2-42dc-a1d1-0e48a5d85aa4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:55.244792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9b26-c7b2-42dc-a1d1-0e48a5d85aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:55.244792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9b26-c7b2-42dc-a1d1-0e48a5d85aa4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:55.246077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 580/1162] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:55.252006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:55.252498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:55.252596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:55.271309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:55.272545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:55.288319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:55.342662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 583/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:55.616179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b9b26-c7b2-42dc-a1d1-0e48a5d85aa4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:55.721846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b9b26-c7b2-42dc-a1d1-0e48a5d85aa4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:55.756451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:55.757060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:55.757337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:55.757607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:55.797761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:55.827496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:55.828991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:55.926183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 581/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:33:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:33:55.933059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:56.034615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afcd5b-9bec-492c-a6c1-d806a0a26d47 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:56.070102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:56.070733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:56.071624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:56.071909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:56.078730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:56.080210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ea63dbaa-8fc5-4ead-91a3-5cc250738207 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:33:56.092683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:33:56.093976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:33:56.099989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 584/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:33:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:33:56.101027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:56.173389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:56.206253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8d96-15ff-4990-9752-d55a02031040 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:56.240391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:56.241001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:56.241277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:56.241548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:56.242191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:33:56.242492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:56.243918 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c60d3-c768-4081-a733-c2aa3eb2e1b1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:56.251444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 582/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Fri Oct 28 09:33:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:33:58.424593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e5069-2392-42e1-a700-2ca8de8fbc17 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:58.425135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e5069-2392-42e1-a700-2ca8de8fbc17 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:58.425135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e5069-2392-42e1-a700-2ca8de8fbc17 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:58.426369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 585/1167] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:33:58.432951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878a786-d2c0-46d3-8965-ba86934eabd6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:58.433472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878a786-d2c0-46d3-8965-ba86934eabd6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:58.433590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878a786-d2c0-46d3-8965-ba86934eabd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:58.452250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2878a786-d2c0-46d3-8965-ba86934eabd6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:33:58.453481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2878a786-d2c0-46d3-8965-ba86934eabd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:33:58.521084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 583/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:33:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:33:58.794885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e5069-2392-42e1-a700-2ca8de8fbc17 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:33:58.807380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:33:58.892399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e5069-2392-42e1-a700-2ca8de8fbc17 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:33:58.925173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251a671-7a24-41a1-86bc-fce1a09db124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:33:58.925766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251a671-7a24-41a1-86bc-fce1a09db124 None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:33:58.926035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251a671-7a24-41a1-86bc-fce1a09db124 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:33:58.926301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251a671-7a24-41a1-86bc-fce1a09db124 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:33:58.926948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251a671-7a24-41a1-86bc-fce1a09db124 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:33:58.928316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251a671-7a24-41a1-86bc-fce1a09db124 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:33:58.943660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 586/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Fri Oct 28 09:33:58 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:34:01.093365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef832-a84b-4729-aad4-fe1128077f98 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:01.093962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef832-a84b-4729-aad4-fe1128077f98 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:01.094056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ef832-a84b-4729-aad4-fe1128077f98 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:01.095524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 584/1170] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:34:01.100700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:01.101112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:01.101248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:01.119514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:34:01.120736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:34:01.183272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:01.188424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 587/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:34:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:34:01.456733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef832-a84b-4729-aad4-fe1128077f98 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:01.519992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:01.558792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ef832-a84b-4729-aad4-fe1128077f98 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:01.592698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec9e0-79ba-43fa-bfb6-c1035536f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:01.593320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec9e0-79ba-43fa-bfb6-c1035536f118 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:01.593597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec9e0-79ba-43fa-bfb6-c1035536f118 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:01.593868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec9e0-79ba-43fa-bfb6-c1035536f118 None admin] PATH_INFO: `/v3/domains/ba918295d8154193b67c0143deffafc1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:01.604297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec9e0-79ba-43fa-bfb6-c1035536f118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ba918295d8154193b67c0143deffafc1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:01.605757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec9e0-79ba-43fa-bfb6-c1035536f118 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:01.607376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 585/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Fri Oct 28 09:34:01 2022] GET /identity/v3/domains/ba918295d8154193b67c0143deffafc1 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:01.613433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:01.710878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bdbb1-4fe5-4b5b-b24e-7ba75fe91530 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:01.745124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc680ee7-35d6-4d95-a482-af505601bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:01.745736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc680ee7-35d6-4d95-a482-af505601bfaa None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:01.746011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc680ee7-35d6-4d95-a482-af505601bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:01.746278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc680ee7-35d6-4d95-a482-af505601bfaa None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:01.746920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc680ee7-35d6-4d95-a482-af505601bfaa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:01.748381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc680ee7-35d6-4d95-a482-af505601bfaa None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:01.776702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 588/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:34:01 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:34:03.996763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7f768-676a-4681-a4bf-d5437595eecf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:03.997325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7f768-676a-4681-a4bf-d5437595eecf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:03.997325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7f768-676a-4681-a4bf-d5437595eecf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:03.998625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 586/1174] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:34:04.004019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:04.004404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:04.004556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:04.022909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:34:04.024200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:34:04.071809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:04.091653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 589/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:34:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:34:04.360747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7f768-676a-4681-a4bf-d5437595eecf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:04.460109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7f768-676a-4681-a4bf-d5437595eecf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:04.483473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:04.494364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e7ef1-e1e9-4f1d-87ca-f57053367ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:04.494990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e7ef1-e1e9-4f1d-87ca-f57053367ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:04.495260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e7ef1-e1e9-4f1d-87ca-f57053367ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:04.495529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e7ef1-e1e9-4f1d-87ca-f57053367ba4 None admin] PATH_INFO: `/v3/domains/ba918295d8154193b67c0143deffafc1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:04.496667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e7ef1-e1e9-4f1d-87ca-f57053367ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ba918295d8154193b67c0143deffafc1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:04.498101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e7ef1-e1e9-4f1d-87ca-f57053367ba4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:04.499659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 587/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Fri Oct 28 09:34:04 2022] GET /identity/v3/domains/ba918295d8154193b67c0143deffafc1 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:04.505470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:04.602327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c29e18-70e8-4fa5-ae5d-2c1eff509fe0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:04.636198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ae229-7bf3-44d1-be3c-78a81f9025ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:04.636726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ae229-7bf3-44d1-be3c-78a81f9025ed None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:04.636996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ae229-7bf3-44d1-be3c-78a81f9025ed None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:04.637262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ae229-7bf3-44d1-be3c-78a81f9025ed None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:04.637910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ae229-7bf3-44d1-be3c-78a81f9025ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:04.639326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ae229-7bf3-44d1-be3c-78a81f9025ed None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:04.661460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 590/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:34:04 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:34:06.857485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63a6a0-ca6f-4d86-a625-c7a6d57e39d2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:06.858041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63a6a0-ca6f-4d86-a625-c7a6d57e39d2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:06.858041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63a6a0-ca6f-4d86-a625-c7a6d57e39d2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:06.859395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 588/1178] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:34:06.864578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:06.864967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:06.865112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:06.883242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:34:06.884466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:34:06.916256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:06.955850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 591/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:34:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:34:07.230480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63a6a0-ca6f-4d86-a625-c7a6d57e39d2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:07.327320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63a6a0-ca6f-4d86-a625-c7a6d57e39d2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:07.360293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:07.360906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:07.361181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:07.361449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:07.371409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:34:07.376226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:34:07.377339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:34:07.378249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 589/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:34:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:34:07.384079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:07.481245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1ea91-01f6-4753-9712-51ccabd1f4c0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:07.515371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:07.516002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:07.516283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:07.516556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:07.517211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:34:07.517509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:07.518869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:07.527811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 592/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:34:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:07.533947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:07.586026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:07.633019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00737d68-2d81-47d8-8a2e-a27d9c32ba19 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:07.667560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:07.668212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:07.668484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:07.668751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] PATH_INFO: `/v3/users/4093f67082cc4644bc1c53faa0e934e3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:07.684368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] RBAC: Authorizing `identity:get_user(user_id=4093f67082cc4644bc1c53faa0e934e3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:07.685821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:07.687461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 590/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:34:07 2022] GET /identity/v3/users/4093f67082cc4644bc1c53faa0e934e3 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:07.693383 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:07.723589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:07.790531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78afd9d-ead0-43a1-a1ae-24cf6349635d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:07.824408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:07.825016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:07.825279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:07.825544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:07.890723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:07.892205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:07.919810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:07.988119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 593/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:34:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:34:07.994485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:08.101731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c40a1c-ba9a-45d9-83bf-b4d302dbda61 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:08.113412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:08.138074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ab6813-58da-477b-92ff-22a943deb819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:08.138707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6813-58da-477b-92ff-22a943deb819 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:08.138983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6813-58da-477b-92ff-22a943deb819 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:08.139252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ab6813-58da-477b-92ff-22a943deb819 None admin] PATH_INFO: `/v3/projects/438493daabc646dca08dcfd2d91f79cc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:08.140398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab6813-58da-477b-92ff-22a943deb819 None admin] RBAC: Authorizing `identity:get_project(project_id=438493daabc646dca08dcfd2d91f79cc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:08.141841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ab6813-58da-477b-92ff-22a943deb819 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:08.143479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 591/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:34:07 2022] GET /identity/v3/projects/438493daabc646dca08dcfd2d91f79cc => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:08.149723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:08.246507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c255558-1d7e-45b0-aca7-2579e12fd1b4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:08.273636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:08.284990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:08.285624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:08.285902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:08.286178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:08.286925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=438493daabc646dca08dcfd2d91f79cc, user.id=4093f67082cc4644bc1c53faa0e934e3) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:34:08.287232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:08.288735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9725e8a-3ee5-46f8-9c96-6940688f877f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:08.295575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 594/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:34:08 2022] GET /identity/v3/role_assignments?user.id=4093f67082cc4644bc1c53faa0e934e3&scope.project.id=438493daabc646dca08dcfd2d91f79cc&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:10.456501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec278c-8df8-46ca-b297-9eddd77bad94 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:10.457085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec278c-8df8-46ca-b297-9eddd77bad94 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:10.457085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec278c-8df8-46ca-b297-9eddd77bad94 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:10.458367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 592/1186] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:34:10.464461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:10.464861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:10.465000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:10.484522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:34:10.485724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:34:10.551237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 595/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:34:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:34:10.819656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec278c-8df8-46ca-b297-9eddd77bad94 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:10.919126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec278c-8df8-46ca-b297-9eddd77bad94 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:10.954152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:10.954774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:10.955050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:10.955318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:10.965678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:34:10.970436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:34:10.971628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:34:10.972772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 593/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:34:10.977836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:11.081046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932d579-1542-40c3-a67f-0b34c900126e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:11.104527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:11.115089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:11.115695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:11.116006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:11.116281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:11.116940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:34:11.117239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:11.119154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:11.127241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 596/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:11.133189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:11.176001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:11.240107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416dfd-db2b-4851-a952-186f8d1b0aea None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:11.276000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:11.276641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:11.276915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:11.277184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] PATH_INFO: `/v3/users/4093f67082cc4644bc1c53faa0e934e3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:11.278351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] RBAC: Authorizing `identity:get_user(user_id=4093f67082cc4644bc1c53faa0e934e3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:11.279835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:11.281474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 594/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:34:11 2022] GET /identity/v3/users/4093f67082cc4644bc1c53faa0e934e3 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:11.287441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:11.388418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927cf14-966d-41b4-a708-50827c11bf4a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:11.420834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f63ef5-bd9f-468e-aeb3-c65aa4146be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:11.421448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f63ef5-bd9f-468e-aeb3-c65aa4146be7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:11.421719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f63ef5-bd9f-468e-aeb3-c65aa4146be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:11.421982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f63ef5-bd9f-468e-aeb3-c65aa4146be7 None admin] PATH_INFO: `/v3/projects/438493daabc646dca08dcfd2d91f79cc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:11.423058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f63ef5-bd9f-468e-aeb3-c65aa4146be7 None admin] RBAC: Authorizing `identity:get_project(project_id=438493daabc646dca08dcfd2d91f79cc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:11.424494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f63ef5-bd9f-468e-aeb3-c65aa4146be7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:11.426110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 597/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:34:11 2022] GET /identity/v3/projects/438493daabc646dca08dcfd2d91f79cc => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:11.433004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:11.529207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97592703-5186-4d24-9dab-2ab29f16953e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:11.536167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:11.562851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae1101-1d84-414f-9713-7d5b5122eebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:11.563502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae1101-1d84-414f-9713-7d5b5122eebe None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:11.563806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae1101-1d84-414f-9713-7d5b5122eebe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:11.564080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae1101-1d84-414f-9713-7d5b5122eebe None admin] PATH_INFO: `/v3/projects/438493daabc646dca08dcfd2d91f79cc/users/4093f67082cc4644bc1c53faa0e934e3/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:11.565948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae1101-1d84-414f-9713-7d5b5122eebe None admin] RBAC: Authorizing `identity:create_grant(project_id=438493daabc646dca08dcfd2d91f79cc, user_id=4093f67082cc4644bc1c53faa0e934e3, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:11.567374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae1101-1d84-414f-9713-7d5b5122eebe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:11.579535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 595/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:34:11 2022] PUT /identity/v3/projects/438493daabc646dca08dcfd2d91f79cc/users/4093f67082cc4644bc1c53faa0e934e3/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:34:13.717215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe40547-8f65-4aca-b14d-add0b9a679e8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:13.717780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe40547-8f65-4aca-b14d-add0b9a679e8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:13.717780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe40547-8f65-4aca-b14d-add0b9a679e8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:13.719103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 598/1193] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:34:13.725693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:13.726155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:13.726254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:13.744589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:34:13.745769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:34:13.812237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 596/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:34:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:34:14.084678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe40547-8f65-4aca-b14d-add0b9a679e8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:14.145084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:14.192142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe40547-8f65-4aca-b14d-add0b9a679e8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:14.225428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:14.226039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:14.226308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:14.226573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:14.237176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:34:14.241996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:34:14.247613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:34:14.248686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 599/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:34:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:34:14.250056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:14.340920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:14.347464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112b620-c29c-4949-a2ca-728dafc83bdf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:14.382157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:14.382761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:14.383033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:14.383351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:14.384033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:34:14.384324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:14.385688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:14.393647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 597/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:34:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:14.399970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:14.496799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a11d25-be60-4b4a-acef-ee6173a3e4a4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:14.510416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:14.530233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:14.530869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:14.531141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:14.531412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] PATH_INFO: `/v3/users/4093f67082cc4644bc1c53faa0e934e3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:14.532558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] RBAC: Authorizing `identity:get_user(user_id=4093f67082cc4644bc1c53faa0e934e3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:14.533977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:14.535541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 600/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Fri Oct 28 09:34:14 2022] GET /identity/v3/users/4093f67082cc4644bc1c53faa0e934e3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:14.542054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:14.654478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02760e1e-d3e3-4d61-8754-d586e56bc63a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:14.687031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:14.687629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:14.687934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:14.688207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:14.714455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:14.754355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:14.755840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:14.851146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 598/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:34:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:34:14.857162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:14.955627 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99557da-3af8-442c-acbe-6ca955725484 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:14.988336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5642876-3515-449c-824a-92f680e6a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:14.988961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5642876-3515-449c-824a-92f680e6a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:14.989243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5642876-3515-449c-824a-92f680e6a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:14.989516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5642876-3515-449c-824a-92f680e6a6d7 None admin] PATH_INFO: `/v3/projects/438493daabc646dca08dcfd2d91f79cc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:14.990616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5642876-3515-449c-824a-92f680e6a6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=438493daabc646dca08dcfd2d91f79cc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:14.992063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5642876-3515-449c-824a-92f680e6a6d7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:14.993707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 601/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Fri Oct 28 09:34:14 2022] GET /identity/v3/projects/438493daabc646dca08dcfd2d91f79cc => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:14.999833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:34:15.055667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:15.097225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f39a5-2e4a-4fac-855e-f36b320c7cfc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:34:15.130283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:34:15.130883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:34:15.131160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:34:15.131428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:34:15.132182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=438493daabc646dca08dcfd2d91f79cc, user.id=4093f67082cc4644bc1c53faa0e934e3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:34:15.132485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:34:15.133911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919820b-7d97-41cf-b3ac-6012b4eba529 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:34:15.139758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 599/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:34:14 2022] GET /identity/v3/role_assignments?user.id=4093f67082cc4644bc1c53faa0e934e3&scope.project.id=438493daabc646dca08dcfd2d91f79cc&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:34:24.984147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=75 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:34:25.137121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=39 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:11.839472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235623a9-c7cc-4c80-a3f4-f2b3991589bf None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:11.840163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235623a9-c7cc-4c80-a3f4-f2b3991589bf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:11.840163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235623a9-c7cc-4c80-a3f4-f2b3991589bf None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:11.841870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 602/1201] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:11.847045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:11.847768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:11.847768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:11.871796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:11.877917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:11.964328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 600/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:35:12.279756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235623a9-c7cc-4c80-a3f4-f2b3991589bf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:12.411444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235623a9-c7cc-4c80-a3f4-f2b3991589bf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:12.470953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:12.471643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:12.472069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:12.472420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:12.486120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:12.487902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:12.508374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:12.509475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:12.510466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:12.510466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:12.510466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:12.510466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 603/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:12.521140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:12.652612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0387a57-7f92-4d99-ac10-31767184d3ce None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:12.688474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:12.697370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:12.697971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:12.698276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:12.698492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:12.699122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:12.699410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:12.701066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887e173-5343-4405-8b80-0fc54fde0ad6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:12.719812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 601/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:12.721910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:12.788119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:12.845041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd3ea9-94cd-4ee8-aa47-f4315ed137c2 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:12.887888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e757a-4d9c-41e2-9c16-bdc1b8919afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:12.888520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e757a-4d9c-41e2-9c16-bdc1b8919afc None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:12.888799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e757a-4d9c-41e2-9c16-bdc1b8919afc None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:12.889071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e757a-4d9c-41e2-9c16-bdc1b8919afc None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:12.889760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e757a-4d9c-41e2-9c16-bdc1b8919afc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:12.891656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e757a-4d9c-41e2-9c16-bdc1b8919afc None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:12.923950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 604/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Fri Oct 28 09:35:12 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:35:15.343783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96208247-8cf7-4f30-8c07-1b2317da6c54 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:15.344325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96208247-8cf7-4f30-8c07-1b2317da6c54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:15.344325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96208247-8cf7-4f30-8c07-1b2317da6c54 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:15.345929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 602/1206] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:15.351112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:15.351606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:15.351709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:15.372453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:15.373654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:15.455229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 605/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:15.738188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96208247-8cf7-4f30-8c07-1b2317da6c54 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:15.858265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96208247-8cf7-4f30-8c07-1b2317da6c54 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:15.897547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:15.898265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:15.898540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:15.898808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:15.909583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:35:15.914895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:15.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:15.920846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 603/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:35:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:15.922620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:15.976814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:16.048297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afc825-af31-4bb5-a9a3-228887aa01c5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:16.094469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:16.095294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:16.095684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:16.095988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:16.096705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:16.097255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:16.098959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:16.110136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 606/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:35:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:16.116403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:16.181783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:16.251452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4678-928e-4809-a6e4-65b8e2b68461 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:16.294724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:16.295426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:16.295711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:16.296031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:16.308678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:16.310362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:16.329134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:16.330239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:16.336727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:16.336727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:16.336727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:16.336727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 604/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:16.337247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:16.463587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:16.466761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503f918-71be-4021-bdc5-0220f73a6fb4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:16.504502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:16.505112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:16.506812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:16.506812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:16.506812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:16.506812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:16.508827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:16.522683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 607/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:16.529585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:16.658220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9457aa-6090-4919-884e-21e2de6c0bb4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:16.660123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:16.699785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:16.700668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:16.701010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:16.701332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:16.711158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:16.713071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:35:16.726697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:16.728320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:16.730053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 605/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:16.734808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:16.868357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0996e13-82be-4bbd-8415-7c1c0120a8db None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:16.914569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:16.915085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:16.915341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:16.915603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:16.916405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:16.916710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:16.918254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:16.939373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 608/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Oct 28 09:35:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:16.945802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:17.059365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8df16-0334-41a7-9a11-e5bbeaec4d42 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:17.102221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:17.102836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:17.103113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:17.103434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:17.143325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:17.194447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:17.196206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:17.310005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 606/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:35:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:35:17.316411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:17.338374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:17.428521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de4da8-f8c2-4f92-bd5d-f9acf6c791c7 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:17.471057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:17.471713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:17.471992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:17.472174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:17.484943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:17.486410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:17.504571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:17.505747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:17.506783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:17.506783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:17.506783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:17.511379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 609/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:17.512667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:17.582889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:17.636575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b6f4f-03d1-4250-a6d9-a16a653ebcef None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:17.679511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:17.680081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:17.680352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:17.680616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:17.681262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:17.681557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:17.682938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:17.696054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 607/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:17.702341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:17.788011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:17.826677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76c261-a7ea-4f50-b3c6-f385e78c5a67 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:17.865673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:17.866311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:17.867225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:17.867488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:17.874909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:17.876380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:35:17.888446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:17.889759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:17.890803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 610/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:35:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:17.895638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:18.015494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8122915-0180-4a21-a25a-15a078eb3441 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:18.047952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:18.065301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:18.066063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:18.066368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:18.066610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:18.067296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:18.067595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:18.069093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cffcc-f37a-4b02-b26a-2d8b2856ab43 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:18.082856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 608/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:35:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:18.089317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:18.201180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483301f-0c74-4b61-b339-34e17bd37647 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:18.237903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:18.244768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:18.245375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:18.245645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:18.245912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:18.246643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=daf577de783b4e8a9bc9c81dd2a2bd35) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:18.246947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:18.248402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5924614-3c8f-4962-8758-45ca067bdfea None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:18.255550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 611/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:35:18 2022] GET /identity/v3/role_assignments?user.id=daf577de783b4e8a9bc9c81dd2a2bd35&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:20.435251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9e262-a278-4510-8e74-b48b45dcd550 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:20.435825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9e262-a278-4510-8e74-b48b45dcd550 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:20.435825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9e262-a278-4510-8e74-b48b45dcd550 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:20.437236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 609/1220] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:20.442740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:20.443195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:20.443287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:20.461855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:20.463419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:20.534278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 612/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:35:20.805324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9e262-a278-4510-8e74-b48b45dcd550 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:20.919220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9e262-a278-4510-8e74-b48b45dcd550 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:20.957954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:20.958613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:20.958889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:20.959157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:20.971007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:35:20.976433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:20.982436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:20.983520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 610/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:35:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:20.984508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:21.095804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3c1e2-b94a-4018-a5bd-6a8d4ccf18ed None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:21.124952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:21.136315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:21.137125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:21.137449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:21.137824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:21.138581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] RBAC: Adding query filter params (name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:21.138978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:21.140539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:21.151787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 613/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:35:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:21.157413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:21.208014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:21.273518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53801f4b-3f95-4bdc-ad17-7630d65dd4e2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:21.310050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:21.310707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:21.310983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:21.311252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:21.323839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:21.325305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:21.341371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:21.342591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:21.343636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:21.343636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:21.343636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:21.343636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 611/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:21.348555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:21.488977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05b38f-6c00-4524-9543-323225f5363b None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:21.528199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:21.529087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:21.529391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:21.529659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:21.530302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:21.530624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:21.532061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:21.545774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 614/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:21.551928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:21.654323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:21.662536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284e794-c286-47f5-b2e3-6e0549ce34e1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:21.697960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:21.698596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:21.698869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:21.699138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:21.706775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:21.708257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:35:21.720927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:21.722110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:21.723087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 612/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:21.728190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:21.783615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:21.847179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f9372-81ce-481d-80ef-5bc62eb9f139 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:21.885339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:21.885943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:21.886214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:21.886502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:21.887255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:21.887616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:21.889209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:21.913432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 615/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Oct 28 09:35:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:21.914849 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:22.036165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b7ebe-903d-4373-9b61-ac519c051719 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:22.077141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:22.077775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:22.078053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:22.078324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:22.089789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:22.091418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:22.106350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:22.107547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:22.109048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:22.109048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:22.109048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:22.109048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 613/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:22.114019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:22.237930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3eec-4381-4b6a-bc04-c3d3c66cd547 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:22.240511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:22.279271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:22.279973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:22.280279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:22.280551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:22.281214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:22.281504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:22.282869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:22.294903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 616/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:22.300816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:22.310662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:22.419054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e832a-0a61-4046-a97e-81297fe188e9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:22.459916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:22.460671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:22.461004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:22.461344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:22.469651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:22.471285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:35:22.484941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:22.485971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:22.487451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 614/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:35:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:35:22.492555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:22.604897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8bad0-aeed-4c7e-a09f-8d0c0d6a3c4c None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:22.645945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:22.646652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:22.646934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:22.647235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:22.648089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:22.648393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:22.649986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08380e10-f17f-4826-b263-b0c050efb339 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:22.669135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 617/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:35:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:22.670628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:22.738856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:22.777014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8315f0-b130-4ef9-9635-ff413db96e71 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:22.822665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a78b90-c985-45ee-b0da-9dd75502f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:22.823246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78b90-c985-45ee-b0da-9dd75502f582 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:22.823519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78b90-c985-45ee-b0da-9dd75502f582 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:22.823819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78b90-c985-45ee-b0da-9dd75502f582 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/daf577de783b4e8a9bc9c81dd2a2bd35/roles/8d45844a36214d29a889ad5e120d00af` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:22.839982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a78b90-c985-45ee-b0da-9dd75502f582 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=daf577de783b4e8a9bc9c81dd2a2bd35, role_id=8d45844a36214d29a889ad5e120d00af)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:22.841438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a78b90-c985-45ee-b0da-9dd75502f582 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:22.853983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 615/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:35:22 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/daf577de783b4e8a9bc9c81dd2a2bd35/roles/8d45844a36214d29a889ad5e120d00af => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:35:25.073945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92980c-76ed-4774-9f11-45e3aa30c85c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:25.074776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92980c-76ed-4774-9f11-45e3aa30c85c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:25.074776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92980c-76ed-4774-9f11-45e3aa30c85c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:25.077037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 618/1233] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:25.082849 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:25.083388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:25.083388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:25.105213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:25.106423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:25.185620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 616/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:25.503850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92980c-76ed-4774-9f11-45e3aa30c85c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:25.533152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:25.628919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92980c-76ed-4774-9f11-45e3aa30c85c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:25.669768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:25.670413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:25.670690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:25.670961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:25.684588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:35:25.690355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:25.692286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:25.693145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 619/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Fri Oct 28 09:35:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:25.700004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:25.757998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:25.832396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335a6af-777e-4b6f-916b-f50c6c5016b7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:25.875472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:25.876102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:25.876376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:25.876644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:25.877303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:25.877600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:25.878982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:25.887991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 617/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Fri Oct 28 09:35:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:35:25.893845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:26.008258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:26.028006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40912f0b-7f56-40fd-bfac-78415d3ccc1f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:26.072316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:26.072316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:26.072316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:26.072316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:26.081850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:26.083438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:26.099453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:26.101318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:26.102240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:26.102240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:26.102240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:26.102240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 620/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:26.107545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:26.233936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee2951-ee2b-4edd-a698-f619475f9ad5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:26.244734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:26.276929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:26.277590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:26.278512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:26.278512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:26.278903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:26.279979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:26.281392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:26.296728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 618/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:35:26.303109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:26.426409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea707bc1-4b0b-47f4-8879-ee3881644fe8 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:26.474746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:26.475374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:26.476287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:26.476562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:26.479236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:26.484449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:26.486044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:35:26.500086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:26.501113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:26.502368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 621/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:26.507466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:26.622866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff16e0-ce31-412c-9aba-af8fd767f91c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:26.661189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:26.661823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:26.662152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:26.662419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:26.663246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:26.664035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:26.665463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:26.684228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 619/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Oct 28 09:35:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:26.689981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:26.805272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e01d9f-b339-460d-958f-dbe6d34a81a5 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:26.850121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:26.850900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:26.851200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:26.851500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:26.918848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:26.926143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:26.927831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:27.033875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 622/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:35:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:35:27.041399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:27.072560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:27.161127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc488a-1c84-48ca-8510-043e69c210a9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:27.198202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:27.198819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:27.199126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:27.199558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:27.211914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:27.213835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:27.229774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:27.230930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:27.231876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:27.231876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:27.231876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:27.231876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 620/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:27.237056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:27.350954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:27.358073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7603940c-5ba7-448c-9d18-f41925e8dd4e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:27.399246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:27.400219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:27.400589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:27.400957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:27.401900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:27.402319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:27.404027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:27.420482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 623/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:27.426455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:27.514391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:27.539378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2122b-bb4e-487b-ac3e-912ff46d59b6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:27.574868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:27.575562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:27.575885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:27.576189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:27.583148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:27.584645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:35:27.596343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:27.597638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:27.598590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 621/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:35:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:27.606486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:27.731791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92a4eb-8f56-43f4-b165-8c35696e6ae9 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:27.785541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:27.786091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:27.786495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:27.786956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:27.788987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:27.789415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:27.791810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddd290-1cdd-4ddd-8880-ada27cf39b64 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:27.810613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 624/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:35:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:27.817756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:27.941973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a83f-5871-4887-8421-69ee8fabbc8c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:27.983770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:27.984393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:27.984669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:27.984934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:27.985659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] RBAC: Adding query filter params (role.id=8d45844a36214d29a889ad5e120d00af, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=daf577de783b4e8a9bc9c81dd2a2bd35) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:27.985969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:27.987466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c4ab4-ca1b-404f-be75-0852ae2cecaf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:27.994181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 622/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:35:27 2022] GET /identity/v3/role_assignments?user.id=daf577de783b4e8a9bc9c81dd2a2bd35&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=8d45844a36214d29a889ad5e120d00af => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:30.327483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19da73-a499-4633-973b-6a8d4babfffd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:30.328227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19da73-a499-4633-973b-6a8d4babfffd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:30.328227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19da73-a499-4633-973b-6a8d4babfffd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:30.329949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 625/1247] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:30.335273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:30.335713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:30.335844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:30.342767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:30.356142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:30.357469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:30.433008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 623/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:30.720009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19da73-a499-4633-973b-6a8d4babfffd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:30.740683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:30.849488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19da73-a499-4633-973b-6a8d4babfffd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:30.891204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:30.891866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:30.892137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:30.892406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:30.902915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:35:30.907709 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:30.908746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:30.909583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 626/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:35:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:30.915541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:31.030300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98931ef9-d577-46b0-9549-ff5d09d4d578 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:31.064031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:31.071271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:31.072042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:31.072374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:31.072736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:31.073682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:31.074040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:31.075889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:31.086173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 624/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:35:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:31.093168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:31.187734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:31.215445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49d1e8-ef72-455e-948e-28a7f81a8663 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:31.258913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:31.259662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:31.259974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:31.260242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:31.271272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:31.273310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:31.290290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:31.291572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:31.293051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:31.293051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:31.293051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:31.293051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 627/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:31.297621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:31.409607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d2c17-f408-4704-9779-ece321525835 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:31.464509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:31.465340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:31.466008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:31.466447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:31.467214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:31.467767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:31.469856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:31.493284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 625/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:31.499446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:31.613103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa7a90-4723-4531-9cec-9fa20a18f60d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:31.641533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:31.651123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:31.651788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:31.652071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:31.652337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:31.660759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:31.662344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:35:31.676355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:31.677602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:31.679287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 628/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:31.684245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:31.748045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:31.807107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8cd33-2dd9-4777-97df-ec0ab0562f74 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:31.857710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:31.858500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:31.861142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:31.861142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:31.861142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:31.861142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:31.862372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:31.885082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 626/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Oct 28 09:35:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:31.893354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:32.013229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6cf51-ff35-4485-ab19-b0251db8cf3f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:32.057486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:32.058678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:32.059048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:32.059492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:32.132681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:32.161442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:32.164235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:32.297119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 629/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:35:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:35:32.304723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:32.427240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3bd3a-843b-46e9-ab21-a3f2aa12f2da None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:32.442310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:32.466510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:32.467177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:32.467454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:32.467725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:32.479903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:32.481383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:32.497136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:32.498273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:32.499179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:32.499179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:32.499179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:32.499179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 627/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:32.504364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:32.595605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:32.610105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367266f2-7261-48b6-893c-38f18b5505f4 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:32.645321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:32.645946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:32.646223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:32.646493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:32.647148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:32.647985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:32.649378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:32.667056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 630/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:32.670049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:32.777875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e96d08-eb4f-4967-8099-341729e28849 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:32.820966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:32.821603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:32.821878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:32.822150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:32.831857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:32.833716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:35:32.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:32.848248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:32.849139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:32.850113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 628/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:35:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:32.854407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:32.957832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bf57a-ee27-4c2b-b970-c8df3ef2279d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:32.992541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:32.993163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:32.993439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:32.993711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:32.994406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:32.994706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:32.996154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedae8-b830-4c48-9c9c-c71613f70e1b None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:33.009134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 631/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:35:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:33.015266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:33.041973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:33.116744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc871bd9-9031-4e5b-be4e-105c5464c24f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:33.152155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:33.152765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:33.153044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:33.153315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:33.154036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=daf577de783b4e8a9bc9c81dd2a2bd35) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:33.154342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:33.155794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7bbcd4-fe94-4ced-b5f8-c011a70023ca None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:33.161206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 629/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:35:33 2022] GET /identity/v3/role_assignments?user.id=daf577de783b4e8a9bc9c81dd2a2bd35&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:35.304678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd863a-c117-4a29-896e-ff3bf799ac01 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:35.305192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd863a-c117-4a29-896e-ff3bf799ac01 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:35.305192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd863a-c117-4a29-896e-ff3bf799ac01 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:35.306555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 632/1261] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:35.311773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:35.312177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:35.312313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:35.329856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:35.331473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:35.402887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 630/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:35.675720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd863a-c117-4a29-896e-ff3bf799ac01 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:35.686615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:35.785874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd863a-c117-4a29-896e-ff3bf799ac01 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:35.836379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:35.837284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:35.837666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:35.838592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:35.854781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:35:35.863076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:35.864260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:35:35.865398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:35.866203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 633/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:35:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:35.872360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:35.891039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:35.979466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c65bb-bc1d-4c3a-b86f-8e73226be4e9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.011878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.012471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.012741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.013005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.013646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:36.013939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.015285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.023475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 631/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:35:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:36.029300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:36.094317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:36.125574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71094d4-35e8-411a-9b98-ba989ece1b72 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.160543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.161164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.161437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.161706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.171442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.172892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:36.187399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:36.188565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:36.189809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:36.189809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:36.189809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:36.189809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 634/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:36.194859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:36.272142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:36.291769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c3b1d-441a-4e2c-b83e-f8932d43fa07 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.323416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.324017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.324297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.324567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.325195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:36.325494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.326853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.338856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 632/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:36.344519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:36.442098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636bdc5a-fa03-461a-b67e-a5d2d56ef0eb None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.461147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:36.475060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.476296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.476571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.476835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.483958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.485376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:35:36.496199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:36.497387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:36.498334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 635/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:36.503195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:36.599652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f194e-6a23-4a6d-9d04-d9bb09ac41f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.632928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.633512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.633787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.634055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.634764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:36.635069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.636512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.644777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:36.653895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 633/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:36.659619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:36.760604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad664d-476b-4bbb-8bb5-fc239128391d None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.798110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.798955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.799256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.799614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.812817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.814234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:36.828363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:36.829515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:36.830436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:36.830436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:36.830436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:36.830436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 636/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:36.835918 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:36.930772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa629f0b-72f6-4a33-954c-3cee89e947bf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:36.963623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:36.964254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:36.964529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:36.964796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:36.965427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:36.965729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:36.967091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:36.977949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 634/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:36.983567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:37.014714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:37.080102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f6a68-3c34-4ad5-9bbd-2e90c45c4a35 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:37.111475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:37.112107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:37.112380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:37.112644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:37.119494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:37.120933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:35:37.131597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:37.133145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:37.134231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 637/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:35:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:37.139024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:37.195963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:37.236104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c551d-48b6-4657-9bc6-6bc4f136a077 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:37.268853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:37.269446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:37.269721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:37.269990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:37.270665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:37.270993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:37.272486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da89fd9-0389-4358-bc42-9158fffc24ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:37.283991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 635/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:35:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:37.290478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:37.380018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:37.388700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a4716-32ea-468f-ad5b-067f2c30156f None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:37.420535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03cef0-b610-4674-a5a0-67201cf36149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:37.421167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03cef0-b610-4674-a5a0-67201cf36149 None admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:37.421437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03cef0-b610-4674-a5a0-67201cf36149 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:37.421704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03cef0-b610-4674-a5a0-67201cf36149 None admin] PATH_INFO: `/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/daf577de783b4e8a9bc9c81dd2a2bd35/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:37.423520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03cef0-b610-4674-a5a0-67201cf36149 None admin] RBAC: Authorizing `identity:create_grant(project_id=6419216789dd4a2f85244acb6bbb9bd8, user_id=daf577de783b4e8a9bc9c81dd2a2bd35, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:37.424943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03cef0-b610-4674-a5a0-67201cf36149 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:37.437341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 638/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Fri Oct 28 09:35:37 2022] PUT /identity/v3/projects/6419216789dd4a2f85244acb6bbb9bd8/users/daf577de783b4e8a9bc9c81dd2a2bd35/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:35:39.604120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbc17d-bfb9-43c7-a94a-ce42998b259e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:39.604726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbc17d-bfb9-43c7-a94a-ce42998b259e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:39.604726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbc17d-bfb9-43c7-a94a-ce42998b259e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:39.606162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 636/1274] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:35:39.612083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588f81-8652-4731-9171-33255817c31d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:39.612617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588f81-8652-4731-9171-33255817c31d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:39.612757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3588f81-8652-4731-9171-33255817c31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:39.631373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3588f81-8652-4731-9171-33255817c31d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:39.633158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3588f81-8652-4731-9171-33255817c31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:39.705510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 639/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 28 09:35:39.996197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbc17d-bfb9-43c7-a94a-ce42998b259e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:40.104715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbc17d-bfb9-43c7-a94a-ce42998b259e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:40.124273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:40.142256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:40.142886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:40.143153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:40.143415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:40.155079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 28 09:35:40.160136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:40.161943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:40.164777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 637/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Fri Oct 28 09:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 28 09:35:40.170235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3588f81-8652-4731-9171-33255817c31d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:40.253978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:40.276855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3588f81-8652-4731-9171-33255817c31d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:40.312494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:40.313171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:40.313494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:40.313810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:40.314532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:40.314881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:40.316355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:40.325766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 640/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Fri Oct 28 09:35:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:40.335422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:40.438895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cc331-d184-469d-b3b4-827927d6189b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:40.474341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:40.474971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:40.475243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:40.475554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:40.485890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:40.487338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:40.502983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:40.504859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:40.506557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:40.506557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:40.506557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:40.506557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 638/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:40.510746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:40.613784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813f64-97e3-4338-933c-078b8c048680 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:40.648174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:40.648831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:40.649145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:40.649450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:40.650135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:40.650469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:40.651905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:40.664818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 641/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:40.670694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:40.703916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:40.777789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df74c14-e173-4563-91f6-246dfe89ddc2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:40.816546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:40.817231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:40.817545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:40.817849 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:40.826873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:40.828501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 28 09:35:40.840468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:40.842304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:40.844206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 639/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:40.848985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:40.853047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:40.962333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40070d87-7c67-4759-a048-8786d60c122a None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:41.001225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:41.001888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:41.002210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:41.002529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:41.003325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:41.003674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:41.005199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:41.023629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 642/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Fri Oct 28 09:35:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:35:41.029609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:41.121318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:41.133598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716985f-4056-44d7-aa66-c1eb6de90be1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:41.175548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:41.176233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:41.176614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:41.176942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:41.245582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:41.247120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:41.342203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:41.355062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 640/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Fri Oct 28 09:35:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 28 09:35:41.361463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:41.434438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:41.477078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a006b31-a809-43a0-82cf-8e39ea20b945 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:41.513797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:41.514436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:41.514705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:41.514974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:41.526492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:41.528033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 28 09:35:41.544183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:41.545416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 28 09:35:41.546358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 28 09:35:41.546358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 28 09:35:41.546358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:41.546358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 643/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:41.552005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:41.668444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e2e4-01be-430b-9297-9c728112541b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:41.705674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:41.706378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:41.706540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:41.706805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:41.707444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:41.707772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:41.709158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:41.722050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 641/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Fri Oct 28 09:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:41.728764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:41.839461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c809f-2476-46a9-99ed-4466c8e13c13 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:41.854033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:41.877354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:41.878032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:41.878372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:41.878643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:41.886536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:41.888011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 28 09:35:41.901153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:41.902965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: ERROR keystone.server.flask.application Oct 28 09:35:41.904320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 644/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Fri Oct 28 09:35:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:41.909825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:41.973601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:42.034696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f306e8-3662-4103-95cf-da3598c899c8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:42.071547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:42.072245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:42.072569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:42.072903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:42.073671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:42.074023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:42.075532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb34dee-a047-4d49-9464-a449ef0f69d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:42.090171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 642/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Fri Oct 28 09:35:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:42.096812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:42.216689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002102-0084-419e-84fa-1f71070e6a4e None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:42.252714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:42.253236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:42.253508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:42.253773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:42.254490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] RBAC: Adding query filter params (role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3, scope.project.id=6419216789dd4a2f85244acb6bbb9bd8, user.id=daf577de783b4e8a9bc9c81dd2a2bd35) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:42.254793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:42.256248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3eba08-8862-48ce-a970-deabf0b66d07 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:42.263570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 645/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Fri Oct 28 09:35:42 2022] GET /identity/v3/role_assignments?user.id=daf577de783b4e8a9bc9c81dd2a2bd35&scope.project.id=6419216789dd4a2f85244acb6bbb9bd8&role.id=9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:35:44.653384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5ebce-e7c2-4832-8cb0-266bebf08cde None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:44.653992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5ebce-e7c2-4832-8cb0-266bebf08cde None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:44.653992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5ebce-e7c2-4832-8cb0-266bebf08cde None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:44.655283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 643/1288] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:44.661527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:44.662040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:44.662040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:44.681667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:44.682943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:44.720262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:44.764261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 646/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:45.040154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5ebce-e7c2-4832-8cb0-266bebf08cde None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:45.151729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5ebce-e7c2-4832-8cb0-266bebf08cde None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:45.163983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:45.190612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:45.191254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:45.191533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:45.191841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:45.192431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:45.193818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:45.203275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:45.204618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:45.205900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 644/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:35:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:45.211715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:45.320916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a64ce-f0f1-4007-bc9e-009affb83814 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:45.361616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:45.362282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:45.362562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:45.362835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:45.363473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:45.363798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:45.365201 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773161a-be8e-4abc-afce-9b3dd96885ee None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:45.370466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 647/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Oct 28 09:35:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:45.378642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:45.484222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa840403-5a16-4fa4-9d07-7e0033b2d2af None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:45.519346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:45.519977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:45.520250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:45.520519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:45.521147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:45.521447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:45.523355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273d321-26ee-46bb-a63c-772f3a8a47f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:45.528226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 645/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Oct 28 09:35:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:47.825338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cab383-22ce-485f-9770-d16f81c30b23 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:47.825906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cab383-22ce-485f-9770-d16f81c30b23 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:47.825906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cab383-22ce-485f-9770-d16f81c30b23 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:47.827280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 648/1293] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:35:47.832999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9748-ec5b-435d-82da-376702cce1b7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:47.833440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9748-ec5b-435d-82da-376702cce1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:47.833540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f9748-ec5b-435d-82da-376702cce1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:47.853205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-498f9748-ec5b-435d-82da-376702cce1b7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:47.854524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-498f9748-ec5b-435d-82da-376702cce1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:47.887334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:47.929406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 646/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:48.203613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cab383-22ce-485f-9770-d16f81c30b23 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:48.237283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:48.315265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cab383-22ce-485f-9770-d16f81c30b23 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:48.353195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53a742-fc3b-4953-882a-f873caed94df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:48.353811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53a742-fc3b-4953-882a-f873caed94df None admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:48.354084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53a742-fc3b-4953-882a-f873caed94df None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:48.354350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53a742-fc3b-4953-882a-f873caed94df None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:48.355018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53a742-fc3b-4953-882a-f873caed94df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:48.356421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53a742-fc3b-4953-882a-f873caed94df None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:48.372902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 649/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Fri Oct 28 09:35:48 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:35:50.815085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9e5dc-f0b4-4f9e-96eb-4fc41ea3e1c3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:50.815667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9e5dc-f0b4-4f9e-96eb-4fc41ea3e1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:50.815798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9e5dc-f0b4-4f9e-96eb-4fc41ea3e1c3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:50.817483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 647/1296] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:35:50.824993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:50.825803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:50.825989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:50.849348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:50.850714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:50.934509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 650/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:51.229196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9e5dc-f0b4-4f9e-96eb-4fc41ea3e1c3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:51.342833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9e5dc-f0b4-4f9e-96eb-4fc41ea3e1c3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:51.384512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:51.385186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:51.385466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:51.385742 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:51.386480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:51.388156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:51.399092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:51.400373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:51.401228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 648/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:35:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:51.406634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:51.432097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:51.536087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8b7fa-3f1e-4885-8b48-6e6a42591d19 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:51.573721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:51.574340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:51.574611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:51.574876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:51.575507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:51.575843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:51.577212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:51.582751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 651/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Oct 28 09:35:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:51.592056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:51.624119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:51.703370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af75923-ef35-465c-a78a-ea7ac60aab44 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:51.741291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:51.741901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:51.742174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:51.742435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:51.743130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] RBAC: Adding query filter params (interface=public, service_id=1154c3b88344447b88b9f756c3a56980, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:51.743431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:51.745614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eed673-7ae4-47ef-86d9-369be10a450e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:51.751206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 649/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Fri Oct 28 09:35:51 2022] GET /identity/v3/endpoints?service_id=1154c3b88344447b88b9f756c3a56980&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:51.758177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:51.845780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:51.863637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80dda1-a4b0-4ebe-8e76-65605f7f7700 None admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:51.898147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84817a-09ed-4ff9-8e3b-0697056bfc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:51.898760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84817a-09ed-4ff9-8e3b-0697056bfc52 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:51.899033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84817a-09ed-4ff9-8e3b-0697056bfc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:51.899299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84817a-09ed-4ff9-8e3b-0697056bfc52 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:51.899932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84817a-09ed-4ff9-8e3b-0697056bfc52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:51.901292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84817a-09ed-4ff9-8e3b-0697056bfc52 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:51.908048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 652/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Fri Oct 28 09:35:51 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:35:54.214115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77dce-d47c-4598-b925-f13d3e76f2ed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:54.214794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77dce-d47c-4598-b925-f13d3e76f2ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:54.214794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77dce-d47c-4598-b925-f13d3e76f2ed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:54.216024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 650/1302] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:35:54.222659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:54.223168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:54.223267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:54.244466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:35:54.245869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:35:54.325471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 653/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:35:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 28 09:35:54.605653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77dce-d47c-4598-b925-f13d3e76f2ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:54.706154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:54.717625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77dce-d47c-4598-b925-f13d3e76f2ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:54.754742 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:54.755371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:54.755688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:54.755983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:54.757088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:54.758471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 28 09:35:54.768430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 28 09:35:54.770289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: ERROR keystone.server.flask.application Oct 28 09:35:54.772021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 651/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Fri Oct 28 09:35:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 28 09:35:54.777303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:54.883977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:35:54.908839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5be31-a54c-416f-ae02-c61a603d9671 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:54.946884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:54.947499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:54.947791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:54.948058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] PATH_INFO: `/v3/services` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:54.948681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:35:54.948972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:54.950487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e3a22-2a39-4f94-a68f-739907fed8e8 None admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:54.956670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 654/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Fri Oct 28 09:35:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:35:54.962618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:35:55.078424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae13170-658b-4105-b38f-0ce94eab3005 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:35:55.114164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d2467-1b39-490c-84e6-d6449e576b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:35:55.114776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d2467-1b39-490c-84e6-d6449e576b41 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:35:55.115055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d2467-1b39-490c-84e6-d6449e576b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:35:55.115323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d2467-1b39-490c-84e6-d6449e576b41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:35:55.116759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d2467-1b39-490c-84e6-d6449e576b41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:35:55.118220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d2467-1b39-490c-84e6-d6449e576b41 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:35:55.145798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 652/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:35:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:36:04.953120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=30 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:36:05.139396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:02.650701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163d725-6fee-4bab-83f3-accda38133ae None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:02.651260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163d725-6fee-4bab-83f3-accda38133ae None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:02.651260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163d725-6fee-4bab-83f3-accda38133ae None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:02.656255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 655/1307] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:02.658458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931a88e-7710-42c6-a6ba-aadfadece2d7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:02.658937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931a88e-7710-42c6-a6ba-aadfadece2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:02.659019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931a88e-7710-42c6-a6ba-aadfadece2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:02.742950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1931a88e-7710-42c6-a6ba-aadfadece2d7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:02.744234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1931a88e-7710-42c6-a6ba-aadfadece2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:02.860088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 653/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:37:03.161902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710deed-49f7-4c01-8638-eb7d9e65986f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:03.162795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710deed-49f7-4c01-8638-eb7d9e65986f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:03.163148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710deed-49f7-4c01-8638-eb7d9e65986f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:03.164946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 656/1309] 10.222.0.52 () {54 vars in 964 bytes} [Fri Oct 28 09:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:03.171219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b89c2-7410-4241-be76-dcd506e59af1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:03.171509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b89c2-7410-4241-be76-dcd506e59af1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:03.171815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b89c2-7410-4241-be76-dcd506e59af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:03.257297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3c2b89c2-7410-4241-be76-dcd506e59af1 None None] MFA Rules not processed for user `b07b6ea0a9a7475fbcc8d6baf537cc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:03.258496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3c2b89c2-7410-4241-be76-dcd506e59af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:03.283557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:03.345199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 654/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Fri Oct 28 09:37:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:03.354973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710deed-49f7-4c01-8638-eb7d9e65986f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:03.356830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8710deed-49f7-4c01-8638-eb7d9e65986f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:03.484556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710deed-49f7-4c01-8638-eb7d9e65986f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:03.564338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:03.564941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:03.565220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:03.565490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:03.567087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:03.634091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:03.704651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:03.706137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad135ca7-8bd4-4812-8438-38e1999728fb service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:03.730387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 657/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:06.213253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cb6ce-57a6-44b8-ba3f-97f490482ed8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:06.213761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cb6ce-57a6-44b8-ba3f-97f490482ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:06.213761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cb6ce-57a6-44b8-ba3f-97f490482ed8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:06.215111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 655/1312] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:06.221046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e14a1-8e53-403c-8b3a-29230f744ba9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:06.221479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e14a1-8e53-403c-8b3a-29230f744ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:06.221642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e14a1-8e53-403c-8b3a-29230f744ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:06.310474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7f8e14a1-8e53-403c-8b3a-29230f744ba9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:06.311703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7f8e14a1-8e53-403c-8b3a-29230f744ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:06.415757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 658/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:06.716040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45cab-0b77-4396-a2f7-f2449dc39755 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:06.716455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45cab-0b77-4396-a2f7-f2449dc39755 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:06.716804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f45cab-0b77-4396-a2f7-f2449dc39755 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:06.718251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 656/1314] 10.222.0.52 () {54 vars in 964 bytes} [Fri Oct 28 09:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:06.724302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d323f-568b-4402-9c33-f343f514c560 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:06.725286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d323f-568b-4402-9c33-f343f514c560 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:06.725639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d323f-568b-4402-9c33-f343f514c560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:06.737356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:06.806480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-006d323f-568b-4402-9c33-f343f514c560 None None] MFA Rules not processed for user `b07b6ea0a9a7475fbcc8d6baf537cc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:06.807694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-006d323f-568b-4402-9c33-f343f514c560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:06.882898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 659/1315] 10.222.0.52 () {58 vars in 1061 bytes} [Fri Oct 28 09:37:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:06.892110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45cab-0b77-4396-a2f7-f2449dc39755 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:06.893635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-22f45cab-0b77-4396-a2f7-f2449dc39755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:07.000870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f45cab-0b77-4396-a2f7-f2449dc39755 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:07.098050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:07.101986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:07.102804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:07.103281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:07.103558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:07.105282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:07.228777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:07.230239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5996bc-14eb-40b0-a2c8-2a945efc576c service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:07.250001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 657/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:10.001917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2542b7-f23e-4552-8825-eb2322e15639 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:10.002499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2542b7-f23e-4552-8825-eb2322e15639 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:10.002499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2542b7-f23e-4552-8825-eb2322e15639 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:10.003822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 660/1317] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:10.009123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d6859-a90e-40d9-a1a3-2debc9163284 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:10.009561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d6859-a90e-40d9-a1a3-2debc9163284 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:10.009668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d6859-a90e-40d9-a1a3-2debc9163284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:10.076959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-211d6859-a90e-40d9-a1a3-2debc9163284 None None] MFA Rules not processed for user `7f8be41948d446bead6b1ad550c58d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:10.078160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-211d6859-a90e-40d9-a1a3-2debc9163284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:10.130145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:10.176229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 658/1318] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:10.463874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2542b7-f23e-4552-8825-eb2322e15639 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:10.465423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1e2542b7-f23e-4552-8825-eb2322e15639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:10.569289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2542b7-f23e-4552-8825-eb2322e15639 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:10.572110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:10.641929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:10.642518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:10.642788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:10.643053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:10.644658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:10.750405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:10.751768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca71178-129d-4890-82c7-ac6fde4375df service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:10.769380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 661/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:13.508889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee599-7fcd-47c6-93f9-02f345538640 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:13.509382 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee599-7fcd-47c6-93f9-02f345538640 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:13.509474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ee599-7fcd-47c6-93f9-02f345538640 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:13.510757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 659/1320] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:13.516102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32495f4-a2ea-4969-bf09-f40cbebe51a0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:13.516532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32495f4-a2ea-4969-bf09-f40cbebe51a0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:13.516648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32495f4-a2ea-4969-bf09-f40cbebe51a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:13.557504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:13.585587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c32495f4-a2ea-4969-bf09-f40cbebe51a0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:13.586887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c32495f4-a2ea-4969-bf09-f40cbebe51a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:13.684518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 662/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:13.972131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee599-7fcd-47c6-93f9-02f345538640 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:13.973649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f66ee599-7fcd-47c6-93f9-02f345538640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:14.090354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ee599-7fcd-47c6-93f9-02f345538640 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:14.168546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:14.169157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:14.169435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:14.169705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:14.171312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:14.178810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:14.279214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:14.280603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93791fe-3f07-4576-8540-118c76223f1d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:14.298923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 660/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:17.142933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c5a4c-e754-4f92-a1ac-af8b38276ccd None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:17.143482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c5a4c-e754-4f92-a1ac-af8b38276ccd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:17.143482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c5a4c-e754-4f92-a1ac-af8b38276ccd None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:17.145236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 663/1323] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:17.150458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a08c6c-535d-4f8c-bf28-052201468592 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:17.150938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a08c6c-535d-4f8c-bf28-052201468592 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:17.151028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a08c6c-535d-4f8c-bf28-052201468592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:17.228654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57a08c6c-535d-4f8c-bf28-052201468592 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:17.229891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57a08c6c-535d-4f8c-bf28-052201468592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:17.295082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:17.338865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 661/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:37:17.624001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c5a4c-e754-4f92-a1ac-af8b38276ccd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:17.625528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-457c5a4c-e754-4f92-a1ac-af8b38276ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:17.730494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c5a4c-e754-4f92-a1ac-af8b38276ccd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:17.743583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:17.805198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:17.805793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:17.806066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:17.806329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:17.807943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:17.921839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:17.923302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d4fd8-37db-4951-a2f6-4ec9a59f2586 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:17.944422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 664/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:20.183238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6b9ed-8b72-4814-bfc9-4726ff3e8ba4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:20.183806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6b9ed-8b72-4814-bfc9-4726ff3e8ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:20.183806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6b9ed-8b72-4814-bfc9-4726ff3e8ba4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:20.185130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 662/1326] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:20.190516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976f86-38d2-4a4d-9a28-983d0f810568 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:20.190934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976f86-38d2-4a4d-9a28-983d0f810568 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:20.191058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c976f86-38d2-4a4d-9a28-983d0f810568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:20.250726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:20.274197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9c976f86-38d2-4a4d-9a28-983d0f810568 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:20.275370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9c976f86-38d2-4a4d-9a28-983d0f810568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:20.419785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 665/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:20.700357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6b9ed-8b72-4814-bfc9-4726ff3e8ba4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:20.702042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fcb6b9ed-8b72-4814-bfc9-4726ff3e8ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:20.812987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6b9ed-8b72-4814-bfc9-4726ff3e8ba4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:20.896825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:20.897433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:20.897713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:20.897970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:20.899581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:20.919691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:21.022821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:21.024295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68798a9b-8f58-4bfb-a8d6-cb9e9e2ead9b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:21.046297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 663/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:23.317287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69934c-0ced-46b1-8b4a-63054d449d6d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:23.317815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69934c-0ced-46b1-8b4a-63054d449d6d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:23.317815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d69934c-0ced-46b1-8b4a-63054d449d6d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:23.319193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 666/1329] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:23.324927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6f3f1-78c8-42f7-be16-35c664edb587 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:23.325408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6f3f1-78c8-42f7-be16-35c664edb587 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:23.325508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6f3f1-78c8-42f7-be16-35c664edb587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:23.394153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04e6f3f1-78c8-42f7-be16-35c664edb587 None None] MFA Rules not processed for user `7f8be41948d446bead6b1ad550c58d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:23.395365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04e6f3f1-78c8-42f7-be16-35c664edb587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:23.464483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:23.496568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 664/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:23.785020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69934c-0ced-46b1-8b4a-63054d449d6d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:23.786883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5d69934c-0ced-46b1-8b4a-63054d449d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:23.918373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d69934c-0ced-46b1-8b4a-63054d449d6d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:23.939706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:23.997028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:23.997630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:23.997902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:23.998165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:23.999762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:24.128136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:24.129471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38deabbd-2959-4f1b-bd59-12062fdc2e91 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:24.150794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 667/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:26.726364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c66622-f318-4497-80a3-39634fc533ab None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:26.726926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c66622-f318-4497-80a3-39634fc533ab None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:26.726926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c66622-f318-4497-80a3-39634fc533ab None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:26.728287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 665/1332] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:37:26.734539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa805a-5584-4390-82b5-2b4b90d7d99b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:26.735061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa805a-5584-4390-82b5-2b4b90d7d99b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:26.735061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa805a-5584-4390-82b5-2b4b90d7d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:26.802950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:26.806286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a8aa805a-5584-4390-82b5-2b4b90d7d99b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:26.808113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a8aa805a-5584-4390-82b5-2b4b90d7d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:26.913326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 668/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:27.192697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c66622-f318-4497-80a3-39634fc533ab None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:27.194291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9c66622-f318-4497-80a3-39634fc533ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:27.302652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c66622-f318-4497-80a3-39634fc533ab None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:27.378274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:27.378885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:27.379157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:27.379421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:27.381116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:27.412833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:27.499164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:27.501238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12057cc9-c764-48df-9561-cb2d6630f2ee service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:27.520729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 666/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:29.787535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08034d-bc1c-4c15-932b-5ef4bc9e18b8 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:29.788117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08034d-bc1c-4c15-932b-5ef4bc9e18b8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:29.788117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08034d-bc1c-4c15-932b-5ef4bc9e18b8 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:29.789378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 669/1335] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:29.794934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22094f2-b6c1-4242-b022-1c5f8ddb1fcc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:29.795374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22094f2-b6c1-4242-b022-1c5f8ddb1fcc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:29.795484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22094f2-b6c1-4242-b022-1c5f8ddb1fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:29.863440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e22094f2-b6c1-4242-b022-1c5f8ddb1fcc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:29.864689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e22094f2-b6c1-4242-b022-1c5f8ddb1fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:29.945088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:29.969959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 667/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:30.358214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08034d-bc1c-4c15-932b-5ef4bc9e18b8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:30.360962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cf08034d-bc1c-4c15-932b-5ef4bc9e18b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:30.512216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08034d-bc1c-4c15-932b-5ef4bc9e18b8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:30.548117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:30.599513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:30.600158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:30.600427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:30.600693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:30.602404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:30.723382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:30.724878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d186d-249e-45d2-b45d-fc1d4747cb96 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:30.743156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 670/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:33.321018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983159c7-b937-4b8e-90c2-6ab8480ccc3a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:33.321814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983159c7-b937-4b8e-90c2-6ab8480ccc3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:33.321814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983159c7-b937-4b8e-90c2-6ab8480ccc3a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:33.322989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 668/1338] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:37:33.329526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897e4f-8be5-4f37-8557-8c0b0e295581 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:33.330161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897e4f-8be5-4f37-8557-8c0b0e295581 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:33.330161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7897e4f-8be5-4f37-8557-8c0b0e295581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:33.409723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b7897e4f-8be5-4f37-8557-8c0b0e295581 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:33.410985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b7897e4f-8be5-4f37-8557-8c0b0e295581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:33.421150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:33.522062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 671/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:37:33.814685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983159c7-b937-4b8e-90c2-6ab8480ccc3a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:33.816412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-983159c7-b937-4b8e-90c2-6ab8480ccc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:33.949346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983159c7-b937-4b8e-90c2-6ab8480ccc3a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:34.031670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:34.032217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:34.032493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:34.032763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:34.034440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:34.074754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:34.167346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:34.168871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18225ecf-bf96-45a3-ab9a-823a41bf2a5d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:34.190014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 669/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:38.735668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ee5ae-b172-48eb-84d3-414e0ad06466 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:38.736425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ee5ae-b172-48eb-84d3-414e0ad06466 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:38.736425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ee5ae-b172-48eb-84d3-414e0ad06466 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:38.737616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 672/1341] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:38.744194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e811cff-2dcd-4cd0-b7fe-f904e29ae231 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:38.744751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e811cff-2dcd-4cd0-b7fe-f904e29ae231 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:38.744751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e811cff-2dcd-4cd0-b7fe-f904e29ae231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:38.815196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8e811cff-2dcd-4cd0-b7fe-f904e29ae231 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:38.816324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8e811cff-2dcd-4cd0-b7fe-f904e29ae231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:38.908819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:38.921412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 670/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:39.208065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ee5ae-b172-48eb-84d3-414e0ad06466 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:39.209707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-451ee5ae-b172-48eb-84d3-414e0ad06466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:39.321014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ee5ae-b172-48eb-84d3-414e0ad06466 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:39.361682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:39.401232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:39.401748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:39.402022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:39.402288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:39.403920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:39.520663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:39.522654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e607a15-776a-429d-8b8e-37ff699ba54f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:39.542580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 673/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:42.426668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4457ec-7518-428a-85e6-1d7438f58137 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:42.427194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4457ec-7518-428a-85e6-1d7438f58137 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:42.427866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4457ec-7518-428a-85e6-1d7438f58137 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:42.429303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 671/1344] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:42.434659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e561b47-acfb-431a-8e32-e4cdc0c39dbd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:42.435132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e561b47-acfb-431a-8e32-e4cdc0c39dbd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:42.435329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e561b47-acfb-431a-8e32-e4cdc0c39dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:42.502166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2e561b47-acfb-431a-8e32-e4cdc0c39dbd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:42.503369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2e561b47-acfb-431a-8e32-e4cdc0c39dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:42.522512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:42.600806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 674/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:42.886331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4457ec-7518-428a-85e6-1d7438f58137 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:42.887906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff4457ec-7518-428a-85e6-1d7438f58137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:43.005776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4457ec-7518-428a-85e6-1d7438f58137 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:43.085419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:43.086024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:43.086302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:43.086570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:43.088179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:43.135419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:43.204404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:43.205987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fd756-0aab-45d7-b415-6a863a282793 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:43.226760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 672/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:47.756417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9f7f-624d-4ef8-987e-53ae251d4ae0 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:47.757023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9f7f-624d-4ef8-987e-53ae251d4ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:47.757023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d9f7f-624d-4ef8-987e-53ae251d4ae0 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:47.762717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 675/1347] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:37:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:47.764396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227128f-8b16-4412-b7e8-2a754dd8c7f5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:47.766055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227128f-8b16-4412-b7e8-2a754dd8c7f5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:47.766792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227128f-8b16-4412-b7e8-2a754dd8c7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:47.878508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1227128f-8b16-4412-b7e8-2a754dd8c7f5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:47.879723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1227128f-8b16-4412-b7e8-2a754dd8c7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:48.016080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:48.016445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 673/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:48.301187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9f7f-624d-4ef8-987e-53ae251d4ae0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:37:48.302756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-668d9f7f-624d-4ef8-987e-53ae251d4ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:48.410946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d9f7f-624d-4ef8-987e-53ae251d4ae0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:37:48.465031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:48.494321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:37:48.494948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:48.495225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:48.495495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:48.497214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:48.613142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:37:48.614575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f3819-a778-4488-a056-686b54a15f8f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:37:48.633659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 676/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:37:53.687273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba2722-0422-4926-b1f3-a10937fa5f96 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:53.687934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba2722-0422-4926-b1f3-a10937fa5f96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:53.687934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba2722-0422-4926-b1f3-a10937fa5f96 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:53.689731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 674/1350] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:53.696739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822caa0-0960-4a18-b88a-eb51a1ea4be5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:53.697170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822caa0-0960-4a18-b88a-eb51a1ea4be5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:53.697276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822caa0-0960-4a18-b88a-eb51a1ea4be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:53.828427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7822caa0-0960-4a18-b88a-eb51a1ea4be5 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:53.830227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7822caa0-0960-4a18-b88a-eb51a1ea4be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:53.871945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:54.012125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 677/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:37:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:37:58.748925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851751a-158a-4e9a-b468-79294330f956 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:58.749903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851751a-158a-4e9a-b468-79294330f956 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:58.750410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851751a-158a-4e9a-b468-79294330f956 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:58.751915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 675/1352] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:37:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:37:58.757870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c694bf-86af-4f74-b9ff-42cc66f0f9e8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:37:58.758365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c694bf-86af-4f74-b9ff-42cc66f0f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:37:58.758450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c694bf-86af-4f74-b9ff-42cc66f0f9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:37:58.843118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-90c694bf-86af-4f74-b9ff-42cc66f0f9e8 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:37:58.844520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-90c694bf-86af-4f74-b9ff-42cc66f0f9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:37:58.922513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:37:58.962520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 678/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:37:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:00.374618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184796aa-da3e-45f6-8117-4b95f972de65 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:00.375797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184796aa-da3e-45f6-8117-4b95f972de65 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:00.376189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184796aa-da3e-45f6-8117-4b95f972de65 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:00.377747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 676/1354] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:38:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:00.383324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ffeb7-06ef-4ebe-92e0-0107a40903ed None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:00.383806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ffeb7-06ef-4ebe-92e0-0107a40903ed None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:00.384034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ffeb7-06ef-4ebe-92e0-0107a40903ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:00.463297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2f6ffeb7-06ef-4ebe-92e0-0107a40903ed None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:00.464542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2f6ffeb7-06ef-4ebe-92e0-0107a40903ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:00.579830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 679/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:38:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:04.206960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab625-57d2-4aec-a8e7-713f956de8af None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:04.208320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab625-57d2-4aec-a8e7-713f956de8af None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:04.208781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab625-57d2-4aec-a8e7-713f956de8af None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:04.211395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 677/1356] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:04.217323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35647441-8007-491a-8a20-dd75d90d26de None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:04.218276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35647441-8007-491a-8a20-dd75d90d26de None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:04.218778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35647441-8007-491a-8a20-dd75d90d26de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:04.287331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:04.331204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-35647441-8007-491a-8a20-dd75d90d26de None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:04.332346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-35647441-8007-491a-8a20-dd75d90d26de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:04.481485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 680/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:04.789217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d32573-e59f-4c04-b41c-9bdf50254073 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:04.789796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d32573-e59f-4c04-b41c-9bdf50254073 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:04.790284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d32573-e59f-4c04-b41c-9bdf50254073 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:04.791997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 678/1358] 10.222.0.52 () {54 vars in 960 bytes} [Fri Oct 28 09:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:04.801202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07bc16-bbb0-4a4e-8991-21e565f57071 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:04.801636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07bc16-bbb0-4a4e-8991-21e565f57071 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:04.802308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07bc16-bbb0-4a4e-8991-21e565f57071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:04.912346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fb07bc16-bbb0-4a4e-8991-21e565f57071 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:04.913898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fb07bc16-bbb0-4a4e-8991-21e565f57071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:04.969731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:05.065065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 681/1359] 10.222.0.52 () {58 vars in 1057 bytes} [Fri Oct 28 09:38:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:05.073323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d32573-e59f-4c04-b41c-9bdf50254073 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:05.075160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4d32573-e59f-4c04-b41c-9bdf50254073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:05.254112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d32573-e59f-4c04-b41c-9bdf50254073 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:05.348322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:05.403333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:05.403975 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:05.404255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:05.404523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:05.406401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:05.510090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:05.511717 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84095867-b0fd-4557-8a77-01139627e71c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:05.535817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 679/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:07.929619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19012638-95ba-4bb2-bf9e-55ea67f14f3c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:07.930158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19012638-95ba-4bb2-bf9e-55ea67f14f3c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:07.930158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19012638-95ba-4bb2-bf9e-55ea67f14f3c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:07.940744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 682/1361] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:07.941468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307ad05-4da6-42b0-adf4-b29a09a3c821 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:07.941468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307ad05-4da6-42b0-adf4-b29a09a3c821 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:07.941468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307ad05-4da6-42b0-adf4-b29a09a3c821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:08.020203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7307ad05-4da6-42b0-adf4-b29a09a3c821 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:08.021428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7307ad05-4da6-42b0-adf4-b29a09a3c821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:08.080653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:08.163057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 680/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:08.452323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19012638-95ba-4bb2-bf9e-55ea67f14f3c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:08.454198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-19012638-95ba-4bb2-bf9e-55ea67f14f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:08.645005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19012638-95ba-4bb2-bf9e-55ea67f14f3c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:08.656722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:08.794318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:08.795026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:08.795263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:08.795533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:08.797205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:08.896843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:08.898538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90390a90-a722-4003-9b4f-6b4b6faa671f service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:08.919118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 683/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:11.472049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bc40a-2c9c-4f7c-82a1-ff030999cc56 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:11.472598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bc40a-2c9c-4f7c-82a1-ff030999cc56 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:11.472598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bc40a-2c9c-4f7c-82a1-ff030999cc56 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:11.475526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 681/1364] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:11.481130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704701-42c0-4e8a-a6dc-3def6320bd7e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:11.481754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704701-42c0-4e8a-a6dc-3def6320bd7e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:11.481754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9704701-42c0-4e8a-a6dc-3def6320bd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:11.498388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:11.559420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e9704701-42c0-4e8a-a6dc-3def6320bd7e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:11.560798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e9704701-42c0-4e8a-a6dc-3def6320bd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:11.677951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 684/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:11.972955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ddab-fd20-4d2a-8fe1-197a2b6760de None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:11.973535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ddab-fd20-4d2a-8fe1-197a2b6760de None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:11.973874 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ddab-fd20-4d2a-8fe1-197a2b6760de None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:11.975357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 682/1366] 10.222.0.52 () {54 vars in 960 bytes} [Fri Oct 28 09:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:11.981086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402954-77f2-4f34-bbe6-91a64d7fa762 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:11.981372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402954-77f2-4f34-bbe6-91a64d7fa762 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:11.981780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402954-77f2-4f34-bbe6-91a64d7fa762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:12.079346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e8402954-77f2-4f34-bbe6-91a64d7fa762 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:12.080768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e8402954-77f2-4f34-bbe6-91a64d7fa762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:12.124468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:12.233640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 685/1367] 10.222.0.52 () {58 vars in 1057 bytes} [Fri Oct 28 09:38:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:12.243388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ddab-fd20-4d2a-8fe1-197a2b6760de None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:12.244987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6784ddab-fd20-4d2a-8fe1-197a2b6760de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:12.444507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ddab-fd20-4d2a-8fe1-197a2b6760de None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:12.468633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:12.590476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:12.591249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:12.591249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:12.591515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:12.593767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:12.683983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971f7d7-d7b8-47e2-be2c-94294a1b1f15 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:12.684577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971f7d7-d7b8-47e2-be2c-94294a1b1f15 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:12.684577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971f7d7-d7b8-47e2-be2c-94294a1b1f15 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:12.686065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 686/1368] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:38:12.691470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93aaa4c-5b67-4f04-891e-4fc6ce448815 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:12.692752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93aaa4c-5b67-4f04-891e-4fc6ce448815 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:12.692752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93aaa4c-5b67-4f04-891e-4fc6ce448815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:12.716454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:12.718371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca93245c-fc31-4310-b20f-72bef0b1d10b service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:12.738058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 683/1369] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:12.806791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b93aaa4c-5b67-4f04-891e-4fc6ce448815 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:12.808138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b93aaa4c-5b67-4f04-891e-4fc6ce448815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:12.941263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:12.950855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 687/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:13.253137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08869fb5-dca8-459d-b963-438f38e4b4b8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:13.253407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08869fb5-dca8-459d-b963-438f38e4b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:13.253668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08869fb5-dca8-459d-b963-438f38e4b4b8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:13.255868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 684/1371] 10.222.0.52 () {54 vars in 957 bytes} [Fri Oct 28 09:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:13.261955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62405-cb66-4e4e-a291-57e307016478 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:13.262402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62405-cb66-4e4e-a291-57e307016478 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:13.262813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62405-cb66-4e4e-a291-57e307016478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:13.374390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4ef62405-cb66-4e4e-a291-57e307016478 None None] MFA Rules not processed for user `16332ba5fe89453292d21325b333407a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:13.375569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ef62405-cb66-4e4e-a291-57e307016478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:13.486276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 688/1372] 10.222.0.52 () {58 vars in 1054 bytes} [Fri Oct 28 09:38:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:13.497337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08869fb5-dca8-459d-b963-438f38e4b4b8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:13.498965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08869fb5-dca8-459d-b963-438f38e4b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:13.531939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:13.633355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08869fb5-dca8-459d-b963-438f38e4b4b8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:13.735394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:13.736035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:13.736298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:13.736644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:13.738156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:13.857036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:13.904656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:13.906268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb4aaa-d7d5-4a72-a88b-e99b3bf583d0 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:13.932573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 685/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:15.589644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3ea31-f7bd-420e-a2aa-0fb22cc5f64f None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:15.590259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3ea31-f7bd-420e-a2aa-0fb22cc5f64f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:15.590259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3ea31-f7bd-420e-a2aa-0fb22cc5f64f None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:15.591520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 689/1374] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:15.597405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b46018-f40c-47cf-813d-904ad15e1049 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:15.597929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b46018-f40c-47cf-813d-904ad15e1049 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:15.598088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b46018-f40c-47cf-813d-904ad15e1049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:15.668387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d5b46018-f40c-47cf-813d-904ad15e1049 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:15.669602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d5b46018-f40c-47cf-813d-904ad15e1049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:15.777699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 686/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:16.073970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3ea31-f7bd-420e-a2aa-0fb22cc5f64f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:16.075608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47c3ea31-f7bd-420e-a2aa-0fb22cc5f64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:16.293720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:16.363080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3ea31-f7bd-420e-a2aa-0fb22cc5f64f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:16.503076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:16.503695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:16.504306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:16.504434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:16.506653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:16.577293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:16.627640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:16.629302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357b0e3a-b1da-4b7b-8b2a-13120d53c249 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:16.650236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 690/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:18.951840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972685-5e61-4117-a1ba-18904fd8da47 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:18.953029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972685-5e61-4117-a1ba-18904fd8da47 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:18.953029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c972685-5e61-4117-a1ba-18904fd8da47 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:18.955856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 687/1377] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:18.960040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890f956-8988-45fc-add1-6a33bdf9f8e5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:18.960736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890f956-8988-45fc-add1-6a33bdf9f8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:18.960736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890f956-8988-45fc-add1-6a33bdf9f8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:19.054362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8890f956-8988-45fc-add1-6a33bdf9f8e5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:19.056432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8890f956-8988-45fc-add1-6a33bdf9f8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:19.195528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 691/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:19.491798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972685-5e61-4117-a1ba-18904fd8da47 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:19.492393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6c972685-5e61-4117-a1ba-18904fd8da47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:19.536133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:19.680020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c972685-5e61-4117-a1ba-18904fd8da47 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:19.846761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:19.853366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:19.854729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:19.855329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:19.855973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:19.857756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:19.967927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:19.969649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859b19a-15a6-4e08-bd7c-6c7545c3bbbf service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:19.994574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 688/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:22.372163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec456c8f-f170-49eb-a692-4d6b6451d999 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:22.372832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec456c8f-f170-49eb-a692-4d6b6451d999 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:22.372832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec456c8f-f170-49eb-a692-4d6b6451d999 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:22.374055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 692/1380] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:22.380713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c888-4c2d-4a9f-968a-798e77c5391b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:22.381265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c888-4c2d-4a9f-968a-798e77c5391b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:22.381364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c888-4c2d-4a9f-968a-798e77c5391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:22.467107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a494c888-4c2d-4a9f-968a-798e77c5391b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:22.468579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a494c888-4c2d-4a9f-968a-798e77c5391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:22.545976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:22.600601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 689/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:22.896766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec456c8f-f170-49eb-a692-4d6b6451d999 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:22.898364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ec456c8f-f170-49eb-a692-4d6b6451d999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:22.936631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:23.142210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec456c8f-f170-49eb-a692-4d6b6451d999 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:23.308199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:23.331698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:23.332350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:23.332716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:23.333017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:23.334749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:23.440663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:23.442248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f83811-4400-4ad1-9106-32068dc75823 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:23.468515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 693/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:26.053042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfd0d-48c6-4566-a6f2-b26fbbd9684d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:26.053591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfd0d-48c6-4566-a6f2-b26fbbd9684d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:26.053768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180cfd0d-48c6-4566-a6f2-b26fbbd9684d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:26.056030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 690/1383] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:26.063199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50cd3-695f-487e-8865-c3f2e8a38683 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:26.064008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50cd3-695f-487e-8865-c3f2e8a38683 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:26.064493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50cd3-695f-487e-8865-c3f2e8a38683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:26.159255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-36f50cd3-695f-487e-8865-c3f2e8a38683 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:26.160741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-36f50cd3-695f-487e-8865-c3f2e8a38683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:26.242644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:26.304551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 694/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:26.601848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfd0d-48c6-4566-a6f2-b26fbbd9684d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:26.604575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-180cfd0d-48c6-4566-a6f2-b26fbbd9684d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:26.809046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180cfd0d-48c6-4566-a6f2-b26fbbd9684d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:26.881111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:26.990784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:26.991767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:26.992152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:26.992524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:26.994970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:27.122364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:27.124064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916168fe-7738-49f9-b0ff-13c4a4886510 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:27.154022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 691/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:29.443362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dae14-d1b8-4219-b0cb-3a2cd4bd23a1 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:29.444240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dae14-d1b8-4219-b0cb-3a2cd4bd23a1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:29.444240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dae14-d1b8-4219-b0cb-3a2cd4bd23a1 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:29.445303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 695/1386] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:38:29.453833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877f79-ccdb-4e41-af9f-cc67f0b66a00 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:29.454373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877f79-ccdb-4e41-af9f-cc67f0b66a00 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:29.454373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da877f79-ccdb-4e41-af9f-cc67f0b66a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:29.539606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da877f79-ccdb-4e41-af9f-cc67f0b66a00 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:29.541251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da877f79-ccdb-4e41-af9f-cc67f0b66a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:29.555782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:29.697177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 692/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:29.966438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0451-401b-449a-af11-31b5a5ff2ab4 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:29.966818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0451-401b-449a-af11-31b5a5ff2ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:29.967522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef0451-401b-449a-af11-31b5a5ff2ab4 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:29.969987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 696/1388] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:29.979672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990eaf77-e7bc-4bfd-b141-c9c23f560139 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:29.980183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990eaf77-e7bc-4bfd-b141-c9c23f560139 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:29.980681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990eaf77-e7bc-4bfd-b141-c9c23f560139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:30.028909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0451-401b-449a-af11-31b5a5ff2ab4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:30.030583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eaef0451-401b-449a-af11-31b5a5ff2ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:30.081449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-990eaf77-e7bc-4bfd-b141-c9c23f560139 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:30.083664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-990eaf77-e7bc-4bfd-b141-c9c23f560139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:30.214214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:30.231714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 693/1389] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:30.243234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef0451-401b-449a-af11-31b5a5ff2ab4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:30.302464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:30.410695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:30.411769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:30.412251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:30.412686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:30.415717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:30.535158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:30.537451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e80e3-2b70-43e0-8c74-5681ab7ebbb7 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:30.560276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 697/1390] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:38:30.621127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a1516-55c0-40de-8cff-b85350d05cc4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:30.621937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a1516-55c0-40de-8cff-b85350d05cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:30.622301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a1516-55c0-40de-8cff-b85350d05cc4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:30.624646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 694/1391] 10.222.0.52 () {54 vars in 962 bytes} [Fri Oct 28 09:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:30.630558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901414cb-6ee3-48a5-964a-16ed3a4ecec1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:30.630825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901414cb-6ee3-48a5-964a-16ed3a4ecec1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:30.631100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901414cb-6ee3-48a5-964a-16ed3a4ecec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:30.708407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:30.733528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-901414cb-6ee3-48a5-964a-16ed3a4ecec1 None None] MFA Rules not processed for user `416d5cf4e87c4ce9928ca7b4f0ee8220`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:30.734842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-901414cb-6ee3-48a5-964a-16ed3a4ecec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:30.829907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 698/1392] 10.222.0.52 () {58 vars in 1059 bytes} [Fri Oct 28 09:38:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:30.841038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a1516-55c0-40de-8cff-b85350d05cc4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:30.842858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-352a1516-55c0-40de-8cff-b85350d05cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:31.009598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a1516-55c0-40de-8cff-b85350d05cc4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:31.090480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:31.090992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:31.091267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:31.091674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:31.093602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:31.140765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:31.220815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:31.222291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3df80-4813-42e1-90e6-6e5b4adf77a5 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:31.242769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 695/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:32.028394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787226f9-8078-4646-8175-a6ae0bd11d9a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:32.029238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787226f9-8078-4646-8175-a6ae0bd11d9a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:32.029238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787226f9-8078-4646-8175-a6ae0bd11d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:32.106647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-787226f9-8078-4646-8175-a6ae0bd11d9a None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:32.107958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-787226f9-8078-4646-8175-a6ae0bd11d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:32.214863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 699/1394] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:38:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:32.269392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf35ad-2825-435a-864d-d984da31bf10 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:32.270379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf35ad-2825-435a-864d-d984da31bf10 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:32.270379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf35ad-2825-435a-864d-d984da31bf10 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:32.272058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 696/1395] 10.222.0.52 () {54 vars in 943 bytes} [Fri Oct 28 09:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:32.280773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad214ecb-b668-4ea1-8932-e55c6ab4b2f0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:32.281251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad214ecb-b668-4ea1-8932-e55c6ab4b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:32.281739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad214ecb-b668-4ea1-8932-e55c6ab4b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:32.299097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:32.394468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad214ecb-b668-4ea1-8932-e55c6ab4b2f0 None None] MFA Rules not processed for user `0cb99f8c316545e2ba685c87006cbfca`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:32.395895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad214ecb-b668-4ea1-8932-e55c6ab4b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:32.539688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 700/1396] 10.222.0.52 () {58 vars in 1040 bytes} [Fri Oct 28 09:38:32 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:32.553337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf35ad-2825-435a-864d-d984da31bf10 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:32.555789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5aaf35ad-2825-435a-864d-d984da31bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:32.750486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf35ad-2825-435a-864d-d984da31bf10 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:32.775286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:32.940478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:32.941455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:32.942475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:32.942958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:32.945982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:33.093033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:33.094703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28066c30-d14e-4a4c-953c-2750cdf48604 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:33.105067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 697/1397] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:38:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:38:33.507844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541a5b4-5163-4b45-8f94-a4db7883096e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:33.508650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541a5b4-5163-4b45-8f94-a4db7883096e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:33.508650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541a5b4-5163-4b45-8f94-a4db7883096e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:33.510193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 701/1398] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:33.517163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7197e7-1f46-424f-9793-8489681c10ae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:33.517917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7197e7-1f46-424f-9793-8489681c10ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:33.518245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7197e7-1f46-424f-9793-8489681c10ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:33.629939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:33.643079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0f7197e7-1f46-424f-9793-8489681c10ae None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:33.644593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f7197e7-1f46-424f-9793-8489681c10ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:33.849436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 698/1399] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:34.169122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541a5b4-5163-4b45-8f94-a4db7883096e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:34.173729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d541a5b4-5163-4b45-8f94-a4db7883096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:34.272263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:34.431736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541a5b4-5163-4b45-8f94-a4db7883096e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:34.649534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:34.650845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:34.651267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:34.651823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:34.655921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:34.724000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:34.816936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:34.820241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c74947-b366-4a9d-9958-7377ca75c8fd service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:34.855987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 702/1400] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:37.180923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b4656-1e20-44a3-9f59-c28733767c5b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:37.181781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b4656-1e20-44a3-9f59-c28733767c5b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:37.181781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b4656-1e20-44a3-9f59-c28733767c5b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:37.185104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 699/1401] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:37.190205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a7a5f-d8cd-425d-ba48-d80ed9c97770 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:37.192575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a7a5f-d8cd-425d-ba48-d80ed9c97770 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:37.193019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a7a5f-d8cd-425d-ba48-d80ed9c97770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:37.296393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b78a7a5f-d8cd-425d-ba48-d80ed9c97770 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:37.297633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b78a7a5f-d8cd-425d-ba48-d80ed9c97770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:37.443068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 703/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:37.744570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b4656-1e20-44a3-9f59-c28733767c5b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:37.746089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bc5b4656-1e20-44a3-9f59-c28733767c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:37.894049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:37.934886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b4656-1e20-44a3-9f59-c28733767c5b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:38.125603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:38.126080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:38.126655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:38.127371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:38.131414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:38.273004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:38.283100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:38.287893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d8b77c-f6a5-4e1d-b8bc-7e5b60cbb53f service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:38.328555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 700/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:40.676579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014905de-b9c7-4425-9054-3e53ca792995 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:40.677227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014905de-b9c7-4425-9054-3e53ca792995 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:40.677227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014905de-b9c7-4425-9054-3e53ca792995 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:40.678848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 704/1404] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:40.690370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe81d-4df4-453e-82b6-4f4d26eb9e96 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:40.690934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe81d-4df4-453e-82b6-4f4d26eb9e96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:40.691582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fe81d-4df4-453e-82b6-4f4d26eb9e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:40.773061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c03fe81d-4df4-453e-82b6-4f4d26eb9e96 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:40.774075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c03fe81d-4df4-453e-82b6-4f4d26eb9e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:40.960508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 701/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:41.496072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014905de-b9c7-4425-9054-3e53ca792995 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:41.497621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-014905de-b9c7-4425-9054-3e53ca792995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:41.501734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:41.741334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014905de-b9c7-4425-9054-3e53ca792995 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:41.874362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:41.931299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:41.932248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:41.932662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:41.933123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:41.935282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:42.069833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:42.072848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059afede-9514-4752-ba87-535af8fc41ec service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:42.109019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 705/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:38:44.433471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12652670-f203-4e6d-afab-0a14c4c39b90 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:44.434013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12652670-f203-4e6d-afab-0a14c4c39b90 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:44.434013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12652670-f203-4e6d-afab-0a14c4c39b90 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:44.435374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 702/1407] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:44.441978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa199c6c-8329-4312-b112-ef54319508d5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:44.443456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa199c6c-8329-4312-b112-ef54319508d5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:44.444042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa199c6c-8329-4312-b112-ef54319508d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:44.567635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-aa199c6c-8329-4312-b112-ef54319508d5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:44.570757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aa199c6c-8329-4312-b112-ef54319508d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:44.639194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:44.715874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 706/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:45.032022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12652670-f203-4e6d-afab-0a14c4c39b90 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:38:45.033597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12652670-f203-4e6d-afab-0a14c4c39b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:45.142688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:45.246815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12652670-f203-4e6d-afab-0a14c4c39b90 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:38:45.392835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:38:45.393583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:45.393864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:45.394135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:45.397276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:45.442058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:45.536653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:38:45.543879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957f2d9-b1f5-41da-b79e-694362024dec service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:38:45.584206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 703/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:38:54.713486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:55.569467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=34 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:56.741962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a590-366b-46e4-82ea-a8d01bb2c600 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:56.742536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a590-366b-46e4-82ea-a8d01bb2c600 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:56.742536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241a590-366b-46e4-82ea-a8d01bb2c600 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:56.743873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 707/1410] 10.222.0.52 () {54 vars in 918 bytes} [Fri Oct 28 09:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:56.749437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ab483-8072-4069-bb9a-3ed923be4680 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:56.749987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ab483-8072-4069-bb9a-3ed923be4680 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:56.749987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ab483-8072-4069-bb9a-3ed923be4680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:56.829909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-738ab483-8072-4069-bb9a-3ed923be4680 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:56.831116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-738ab483-8072-4069-bb9a-3ed923be4680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:56.942142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 704/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Fri Oct 28 09:38:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:38:56.957066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a252fa-8777-41dc-b803-bf03f574b087 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:56.957339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a252fa-8777-41dc-b803-bf03f574b087 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:56.957631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a252fa-8777-41dc-b803-bf03f574b087 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:56.959056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 708/1412] 10.222.0.52 () {54 vars in 918 bytes} [Fri Oct 28 09:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:38:56.965392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c452c-6f32-4b06-b08d-9b81b6c7a9ca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:56.965704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c452c-6f32-4b06-b08d-9b81b6c7a9ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:56.965942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c452c-6f32-4b06-b08d-9b81b6c7a9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:57.067412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:38:57.072277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7d3c452c-6f32-4b06-b08d-9b81b6c7a9ca None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:57.074882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7d3c452c-6f32-4b06-b08d-9b81b6c7a9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:38:57.230989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 705/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Fri Oct 28 09:38:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:38:59.897724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ca7a1-2fc9-48c9-8df3-28c3a45bb939 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:59.898263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ca7a1-2fc9-48c9-8df3-28c3a45bb939 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:59.898263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ca7a1-2fc9-48c9-8df3-28c3a45bb939 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:59.899518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:38:59.905300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7bee4-ddbf-4ef1-979e-6a809284d6a9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:38:59.905775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7bee4-ddbf-4ef1-979e-6a809284d6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:38:59.905889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec7bee4-ddbf-4ef1-979e-6a809284d6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:38:59.976798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3ec7bee4-ddbf-4ef1-979e-6a809284d6a9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:38:59.978023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ec7bee4-ddbf-4ef1-979e-6a809284d6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:00.042445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:00.090532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:39:00.392770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ca7a1-2fc9-48c9-8df3-28c3a45bb939 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:00.395357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-111ca7a1-2fc9-48c9-8df3-28c3a45bb939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:00.596399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ca7a1-2fc9-48c9-8df3-28c3a45bb939 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:00.712592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:00.756187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:00.756794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:00.757066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:00.757331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:00.758935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:00.853045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:00.854550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73385740-88f1-4180-8e20-05e78ba0db5b service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:00.874228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 710/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:02.071080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe793b1-9af6-40bb-839f-f9410b9f73f3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:02.071629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe793b1-9af6-40bb-839f-f9410b9f73f3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:02.071846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe793b1-9af6-40bb-839f-f9410b9f73f3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:02.073733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:02.082716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5348f-71e7-43bc-bb0b-3b3746a60a3a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:02.083384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5348f-71e7-43bc-bb0b-3b3746a60a3a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:02.083384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5348f-71e7-43bc-bb0b-3b3746a60a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:02.173349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f2c5348f-71e7-43bc-bb0b-3b3746a60a3a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:02.174692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2c5348f-71e7-43bc-bb0b-3b3746a60a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:02.204104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:02.284001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:02.447948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe793b1-9af6-40bb-839f-f9410b9f73f3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:02.449483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9fe793b1-9af6-40bb-839f-f9410b9f73f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:02.645267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe793b1-9af6-40bb-839f-f9410b9f73f3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:02.691202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:02.767842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:02.768390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:02.768663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:02.768930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:02.771174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:02.988529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:02.990218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f553403-67cd-4514-80bf-6d0ce008a74d service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:03.010955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 708/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:04.177270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2231e-d1ed-4dd3-8927-e074fdc656a5 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:04.177795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2231e-d1ed-4dd3-8927-e074fdc656a5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:04.177936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2231e-d1ed-4dd3-8927-e074fdc656a5 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:04.179282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:04.185979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a61b1-94dd-46d0-97d4-430e671fb1d0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:04.186482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a61b1-94dd-46d0-97d4-430e671fb1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:04.186580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a61b1-94dd-46d0-97d4-430e671fb1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:04.241097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:04.265618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d1a61b1-94dd-46d0-97d4-430e671fb1d0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:04.266810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d1a61b1-94dd-46d0-97d4-430e671fb1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:04.364390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:04.519145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2231e-d1ed-4dd3-8927-e074fdc656a5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:04.520733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-23e2231e-d1ed-4dd3-8927-e074fdc656a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:04.674193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e2231e-d1ed-4dd3-8927-e074fdc656a5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:04.690513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:04.799190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:04.799823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:04.800094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:04.800421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:04.802045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:04.915545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:04.917199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea1b6a-2eec-4aa3-af0a-04dc0cfc4c9e service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:04.937836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 713/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:39:06.078439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbccc-1a4e-45f6-92d0-d80eb657d031 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:06.079019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbccc-1a4e-45f6-92d0-d80eb657d031 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:06.079019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cbccc-1a4e-45f6-92d0-d80eb657d031 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:06.080367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:06.087803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f28e2c-6e12-4a62-9b20-76b62c4e6c86 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:06.088326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f28e2c-6e12-4a62-9b20-76b62c4e6c86 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:06.088326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f28e2c-6e12-4a62-9b20-76b62c4e6c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:06.109643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:06.171993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-36f28e2c-6e12-4a62-9b20-76b62c4e6c86 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:06.173222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-36f28e2c-6e12-4a62-9b20-76b62c4e6c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:06.274338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:06.423619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cbccc-1a4e-45f6-92d0-d80eb657d031 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:06.426995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-548cbccc-1a4e-45f6-92d0-d80eb657d031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:06.555377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:06.595534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cbccc-1a4e-45f6-92d0-d80eb657d031 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:06.712024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:06.713013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:06.713500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:06.713986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:06.716370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:06.796721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:06.813590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:06.815145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550b82d-3d43-468e-83b0-730a70794fcb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:06.836676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 711/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:07.990503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3c98-ab70-4fda-b24d-45d1bda5d79c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:07.991102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3c98-ab70-4fda-b24d-45d1bda5d79c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:07.991212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3c98-ab70-4fda-b24d-45d1bda5d79c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:07.993236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:39:08.000420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924ceb7-5487-4421-9601-0610b6239e06 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:08.000982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924ceb7-5487-4421-9601-0610b6239e06 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:08.000982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924ceb7-5487-4421-9601-0610b6239e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:08.092092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e924ceb7-5487-4421-9601-0610b6239e06 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:08.093656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e924ceb7-5487-4421-9601-0610b6239e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:08.229898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:39:08.381000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3c98-ab70-4fda-b24d-45d1bda5d79c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:08.382683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c67b3c98-ab70-4fda-b24d-45d1bda5d79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:08.481611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:08.548404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3c98-ab70-4fda-b24d-45d1bda5d79c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:08.668374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:08.668895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:08.669171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:08.669431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:08.671032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:08.724682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:08.770958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:08.772580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c30ac8-7768-4ffb-8ec1-9fd378de2cc1 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:08.792124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 716/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:39:09.885216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc15125-652d-47ea-980b-1a4d2ba0d779 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:09.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc15125-652d-47ea-980b-1a4d2ba0d779 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:09.885757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc15125-652d-47ea-980b-1a4d2ba0d779 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:09.887044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:09.894660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf1157-494c-4fbd-afb6-8cdab323cc8b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:09.895242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf1157-494c-4fbd-afb6-8cdab323cc8b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:09.895242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf1157-494c-4fbd-afb6-8cdab323cc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:09.977905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9bbf1157-494c-4fbd-afb6-8cdab323cc8b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:09.979126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9bbf1157-494c-4fbd-afb6-8cdab323cc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:10.087128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:10.239098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc15125-652d-47ea-980b-1a4d2ba0d779 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:10.240660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fcc15125-652d-47ea-980b-1a4d2ba0d779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:10.292653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:10.425747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc15125-652d-47ea-980b-1a4d2ba0d779 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:10.559944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:10.560560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:10.560829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:10.561096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:10.562710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:10.569914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:10.656350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:10.657955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b288f-3522-432f-8872-b99e08ae8321 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:10.677958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 714/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:11.807414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335d363-ef12-44a8-95f1-2054cc977849 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:11.808064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335d363-ef12-44a8-95f1-2054cc977849 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:11.808064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0335d363-ef12-44a8-95f1-2054cc977849 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:11.809328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:39:11.820104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4fd98-b652-4beb-a841-b21b249fdfaa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:11.820672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4fd98-b652-4beb-a841-b21b249fdfaa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:11.820672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df4fd98-b652-4beb-a841-b21b249fdfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:11.907056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2df4fd98-b652-4beb-a841-b21b249fdfaa None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:11.909168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2df4fd98-b652-4beb-a841-b21b249fdfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:12.000568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:12.045014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:12.198509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335d363-ef12-44a8-95f1-2054cc977849 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:12.200059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0335d363-ef12-44a8-95f1-2054cc977849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:12.219280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:12.356960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0335d363-ef12-44a8-95f1-2054cc977849 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:12.449672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:12.481330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:12.481948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:12.482224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:12.482493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:12.484149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:12.580558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:12.582204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab27c38-8cd8-42ce-b257-5b1d8fff1bcc service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:12.602016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 719/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:13.700841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944174e-cf6e-4f4b-ae30-33182a523dea None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:13.701429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944174e-cf6e-4f4b-ae30-33182a523dea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:13.701429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944174e-cf6e-4f4b-ae30-33182a523dea None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:13.702741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:13.710813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168916d-f369-43f7-a6e6-a93c8617f8bd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:13.711328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168916d-f369-43f7-a6e6-a93c8617f8bd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:13.711890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168916d-f369-43f7-a6e6-a93c8617f8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:13.802736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9168916d-f369-43f7-a6e6-a93c8617f8bd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:13.804002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9168916d-f369-43f7-a6e6-a93c8617f8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:13.848027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:13.905934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:14.059015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944174e-cf6e-4f4b-ae30-33182a523dea None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:14.060590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7944174e-cf6e-4f4b-ae30-33182a523dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:14.242484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944174e-cf6e-4f4b-ae30-33182a523dea None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:14.301149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:14.368353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:14.369431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:14.369922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:14.370409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:14.373638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:14.480704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:14.482355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d563703-bb51-4b54-b6ac-00b092aa2ce2 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:14.504885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 717/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:15.601076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee8d6-50b5-4db6-9ae3-ecfce9aac13c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:15.601632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee8d6-50b5-4db6-9ae3-ecfce9aac13c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:15.601632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ee8d6-50b5-4db6-9ae3-ecfce9aac13c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:15.603154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 28 09:39:15.610661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf17087-0415-48fa-ab1c-e2bb095991dc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:15.611232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf17087-0415-48fa-ab1c-e2bb095991dc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:15.611232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf17087-0415-48fa-ab1c-e2bb095991dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:15.739530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4cf17087-0415-48fa-ab1c-e2bb095991dc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:15.741480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4cf17087-0415-48fa-ab1c-e2bb095991dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:15.745090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:15.868361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:16.019977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee8d6-50b5-4db6-9ae3-ecfce9aac13c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:16.021629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-111ee8d6-50b5-4db6-9ae3-ecfce9aac13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:16.177140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ee8d6-50b5-4db6-9ae3-ecfce9aac13c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:16.205355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:16.311240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:16.312074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:16.312465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:16.312846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:16.314941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:16.412445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:16.413992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98791c46-130c-4c19-b8ec-6cc02857d9bc service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:16.434831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 722/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:39:18.540908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d25aff-7577-4bc3-a390-d51734080637 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:18.541491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d25aff-7577-4bc3-a390-d51734080637 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:18.541491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d25aff-7577-4bc3-a390-d51734080637 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:18.543137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:18.548931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94666-1c3b-4f39-ae2d-80f6b6ac18f0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:18.549686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94666-1c3b-4f39-ae2d-80f6b6ac18f0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:18.550508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94666-1c3b-4f39-ae2d-80f6b6ac18f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:18.592668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:18.619819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e0d94666-1c3b-4f39-ae2d-80f6b6ac18f0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:18.621035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e0d94666-1c3b-4f39-ae2d-80f6b6ac18f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:18.725270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:19.015497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d25aff-7577-4bc3-a390-d51734080637 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:19.017140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8d25aff-7577-4bc3-a390-d51734080637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:19.152432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d25aff-7577-4bc3-a390-d51734080637 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:19.190519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:19.237267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:19.237912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:19.238186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:19.238473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:19.240089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:19.361080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:19.362546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c400a-43ce-4c81-ac43-f8f499e571ef service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:19.381379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 720/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:21.737938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a436cd-83ed-40aa-8cf8-cb7b0e061236 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:21.738504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a436cd-83ed-40aa-8cf8-cb7b0e061236 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:21.738504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a436cd-83ed-40aa-8cf8-cb7b0e061236 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:21.739844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:21.745817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cbbf-6ace-4c98-a038-be40a8668c78 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:21.746427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cbbf-6ace-4c98-a038-be40a8668c78 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:21.746427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594cbbf-6ace-4c98-a038-be40a8668c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:21.815356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a594cbbf-6ace-4c98-a038-be40a8668c78 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:21.817585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a594cbbf-6ace-4c98-a038-be40a8668c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:21.832351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:21.933481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:39:22.220992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a436cd-83ed-40aa-8cf8-cb7b0e061236 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:22.223150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-77a436cd-83ed-40aa-8cf8-cb7b0e061236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:22.331083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a436cd-83ed-40aa-8cf8-cb7b0e061236 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:22.345589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:22.409601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:22.410206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:22.410471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:22.410735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:22.412340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:22.525611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:22.526972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce86a54e-5463-45bf-a85a-02a85cd33cb8 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:22.545055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 725/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:25.138460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e0917-7dcd-4c35-b044-089418700c3f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:25.139041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e0917-7dcd-4c35-b044-089418700c3f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:25.139041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e0917-7dcd-4c35-b044-089418700c3f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:25.140406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:25.146168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58ddee-a4aa-4c0c-90ab-8b02c2316e41 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:25.146545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58ddee-a4aa-4c0c-90ab-8b02c2316e41 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:25.146720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58ddee-a4aa-4c0c-90ab-8b02c2316e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:25.195851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:25.217026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fc58ddee-a4aa-4c0c-90ab-8b02c2316e41 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:25.218535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fc58ddee-a4aa-4c0c-90ab-8b02c2316e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:25.321111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:25.605006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e0917-7dcd-4c35-b044-089418700c3f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:25.606572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db3e0917-7dcd-4c35-b044-089418700c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:25.711001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e0917-7dcd-4c35-b044-089418700c3f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:25.752570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:25.789570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:25.790199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:25.790468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:25.790726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:25.792391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:25.903951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:25.905287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7aa47-5c3b-4b50-a33c-d54fbc4951b0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:25.924440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 723/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:28.244003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6486a-fce3-41a7-b976-180bd3480c02 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:28.244538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6486a-fce3-41a7-b976-180bd3480c02 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:28.244538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6486a-fce3-41a7-b976-180bd3480c02 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:28.245867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:28.251417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627627-9e86-4eef-bcb5-3123b0686108 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:28.251909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627627-9e86-4eef-bcb5-3123b0686108 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:28.252014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1627627-9e86-4eef-bcb5-3123b0686108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:28.321235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f1627627-9e86-4eef-bcb5-3123b0686108 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:28.322449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f1627627-9e86-4eef-bcb5-3123b0686108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:28.343544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:28.419760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:28.706096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6486a-fce3-41a7-b976-180bd3480c02 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:28.707628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e2a6486a-fce3-41a7-b976-180bd3480c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:28.818807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6486a-fce3-41a7-b976-180bd3480c02 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:28.842969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:28.896908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:28.897537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:28.897806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:28.898067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:28.899643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:29.008522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:29.009880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e61e9f-8c06-481b-90fd-8d7391c07e35 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:29.027350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 728/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:31.280675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0facf-74bc-4630-8fc3-418af47db72d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:31.281229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0facf-74bc-4630-8fc3-418af47db72d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:31.281229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0facf-74bc-4630-8fc3-418af47db72d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:31.282509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:31.289234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7224f-6fe5-4b03-b641-ba42d4f63050 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:31.289801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7224f-6fe5-4b03-b641-ba42d4f63050 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:31.289801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7224f-6fe5-4b03-b641-ba42d4f63050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:31.362707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:31.366336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-28e7224f-6fe5-4b03-b641-ba42d4f63050 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:31.368319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-28e7224f-6fe5-4b03-b641-ba42d4f63050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:31.473258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:31.757564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0facf-74bc-4630-8fc3-418af47db72d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:31.759096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04a0facf-74bc-4630-8fc3-418af47db72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:31.863313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0facf-74bc-4630-8fc3-418af47db72d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:31.914512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:31.939193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:31.939840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:31.940118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:31.940387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:31.941989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:32.051547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:32.053033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542da309-20c7-47f6-9f45-cce0de9872ce service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:32.072178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 726/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:34.308387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1799f-7266-499b-bebc-8ce774ff3586 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:34.308927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1799f-7266-499b-bebc-8ce774ff3586 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:34.308927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1799f-7266-499b-bebc-8ce774ff3586 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:34.310247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:34.315913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf468bb-d286-43ab-a787-ec77a551e0f9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:34.316415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf468bb-d286-43ab-a787-ec77a551e0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:34.316522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf468bb-d286-43ab-a787-ec77a551e0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:34.385576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eaf468bb-d286-43ab-a787-ec77a551e0f9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:34.386705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eaf468bb-d286-43ab-a787-ec77a551e0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:34.419243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:34.517123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:34.807267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1799f-7266-499b-bebc-8ce774ff3586 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:34.808842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4b1799f-7266-499b-bebc-8ce774ff3586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:34.913928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1799f-7266-499b-bebc-8ce774ff3586 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:34.943600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:34.989404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:34.990002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:34.990272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:34.990610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:34.992145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:35.101865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:35.103195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f800999-8a71-4217-b111-82f9a4d9bcea service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:35.121387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 731/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:37.366467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c666e-7527-487b-97af-ee514ca5b21a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:37.367031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c666e-7527-487b-97af-ee514ca5b21a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:37.367031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c666e-7527-487b-97af-ee514ca5b21a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:37.368410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:37.373916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d37d-9a26-4886-91b0-a7f6dfecaf14 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:37.374337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d37d-9a26-4886-91b0-a7f6dfecaf14 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:37.374465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2d37d-9a26-4886-91b0-a7f6dfecaf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:37.444675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-51e2d37d-9a26-4886-91b0-a7f6dfecaf14 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:37.445900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-51e2d37d-9a26-4886-91b0-a7f6dfecaf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:37.454502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:37.547840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:37.831411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c666e-7527-487b-97af-ee514ca5b21a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:37.832964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-330c666e-7527-487b-97af-ee514ca5b21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:37.937048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c666e-7527-487b-97af-ee514ca5b21a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:37.994269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:38.014332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:38.014943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:38.015220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:38.015493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:38.017136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:38.137990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:38.139374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd19e9f4-17c7-4a53-8dd2-dab2fe27be80 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:38.158697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 729/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:40.399143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129885-2cd1-4685-ad73-6e842e86216d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:40.399689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129885-2cd1-4685-ad73-6e842e86216d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:40.399689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9129885-2cd1-4685-ad73-6e842e86216d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:40.401006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:40.407041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce76c-ca23-4b05-a89b-a24ad82e548c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:40.407621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce76c-ca23-4b05-a89b-a24ad82e548c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:40.407621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ce76c-ca23-4b05-a89b-a24ad82e548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:40.480296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-129ce76c-ca23-4b05-a89b-a24ad82e548c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:40.481501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-129ce76c-ca23-4b05-a89b-a24ad82e548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:40.522348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:40.582394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:40.868497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129885-2cd1-4685-ad73-6e842e86216d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:40.870099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e9129885-2cd1-4685-ad73-6e842e86216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:40.974612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9129885-2cd1-4685-ad73-6e842e86216d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:41.012804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:41.055345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:41.055991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:41.056266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:41.056534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:41.058177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:41.170868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:41.172349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9064df5d-0eed-44d7-8586-18839d7364ff service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:41.190743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 734/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:43.407373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d491-10e6-42c0-8dc4-3a4cfb793f6a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:43.407952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d491-10e6-42c0-8dc4-3a4cfb793f6a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:43.407952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356d491-10e6-42c0-8dc4-3a4cfb793f6a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:43.409195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:43.414613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876220ad-4e4e-4c4d-bc89-d0cfbf8f8c46 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:43.415067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876220ad-4e4e-4c4d-bc89-d0cfbf8f8c46 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:43.415164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876220ad-4e4e-4c4d-bc89-d0cfbf8f8c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:43.482731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-876220ad-4e4e-4c4d-bc89-d0cfbf8f8c46 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:43.483967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-876220ad-4e4e-4c4d-bc89-d0cfbf8f8c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:43.504768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:43.588303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:43.885087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356d491-10e6-42c0-8dc4-3a4cfb793f6a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:43.886640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5356d491-10e6-42c0-8dc4-3a4cfb793f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:43.995304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356d491-10e6-42c0-8dc4-3a4cfb793f6a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:44.071058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:44.079179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:44.079809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:44.080078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:44.080338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:44.081935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:44.193951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:44.195396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3fa19-10fa-4916-8ee3-bb80559c8ab5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:44.213522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 732/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:46.394045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28134e77-0524-4f08-93f4-5974719c559b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:46.394590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28134e77-0524-4f08-93f4-5974719c559b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:46.394590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28134e77-0524-4f08-93f4-5974719c559b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:46.395875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:39:46.401367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef615a-82b6-4570-b004-2039eca07fbb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:46.401842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef615a-82b6-4570-b004-2039eca07fbb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:46.401957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef615a-82b6-4570-b004-2039eca07fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:46.467200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-53ef615a-82b6-4570-b004-2039eca07fbb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:39:46.468711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-53ef615a-82b6-4570-b004-2039eca07fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:46.517015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:46.566300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:39:46.861619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28134e77-0524-4f08-93f4-5974719c559b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:39:46.863210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-28134e77-0524-4f08-93f4-5974719c559b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:46.967146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28134e77-0524-4f08-93f4-5974719c559b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:39:47.017454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:47.046528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:39:47.047133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:39:47.047412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:39:47.047704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:39:47.049449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:39:47.165475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:39:47.166909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3e87b0-d7f9-4010-96ab-6ab908bb5087 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:39:47.185083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 737/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:39:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:39:56.564469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=22 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:39:57.182540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=66 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:00.957808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c5e5-544b-4c23-aa78-b39134594252 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:00.958365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c5e5-544b-4c23-aa78-b39134594252 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:00.958472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c5e5-544b-4c23-aa78-b39134594252 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:00.959788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 734/1471] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:00.965395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde35e00-47d1-4fb5-903f-b0007a08c45b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:00.966064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde35e00-47d1-4fb5-903f-b0007a08c45b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:00.966064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde35e00-47d1-4fb5-903f-b0007a08c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:01.033826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cde35e00-47d1-4fb5-903f-b0007a08c45b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:01.035033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cde35e00-47d1-4fb5-903f-b0007a08c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:01.132345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 738/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:01.427425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e7bf2-a1c4-4e33-b317-42e236f45c24 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:01.427706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e7bf2-a1c4-4e33-b317-42e236f45c24 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:01.428008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e7bf2-a1c4-4e33-b317-42e236f45c24 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:01.429346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 735/1473] 10.222.0.52 () {54 vars in 962 bytes} [Fri Oct 28 09:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:01.434615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b6388-069b-41c0-89e3-a81df1bad230 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:01.434892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b6388-069b-41c0-89e3-a81df1bad230 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:01.435161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b6388-069b-41c0-89e3-a81df1bad230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:01.511771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b76b6388-069b-41c0-89e3-a81df1bad230 None None] MFA Rules not processed for user `416d5cf4e87c4ce9928ca7b4f0ee8220`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:01.512990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b76b6388-069b-41c0-89e3-a81df1bad230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:01.534407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:01.592242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 739/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Fri Oct 28 09:40:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:01.600952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e7bf2-a1c4-4e33-b317-42e236f45c24 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:01.602461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c05e7bf2-a1c4-4e33-b317-42e236f45c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:01.709769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e7bf2-a1c4-4e33-b317-42e236f45c24 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:01.790363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:01.790973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:01.791242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:01.791507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:01.793174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:01.853499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:01.911407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:01.912886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae550fed-7f54-41b5-8a30-579b82ae87be service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:01.931442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 736/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:40:04.191973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94cb6-4a31-47c8-a955-aab940b83bf7 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:04.192536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94cb6-4a31-47c8-a955-aab940b83bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:04.192536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa94cb6-4a31-47c8-a955-aab940b83bf7 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:04.193883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 740/1476] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:04.199321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78eca-6e77-4576-9349-42dcdc9002cc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:04.199788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78eca-6e77-4576-9349-42dcdc9002cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:04.200024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec78eca-6e77-4576-9349-42dcdc9002cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:04.265592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dec78eca-6e77-4576-9349-42dcdc9002cc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:04.266823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dec78eca-6e77-4576-9349-42dcdc9002cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:04.365876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 737/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:04.638494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa94cb6-4a31-47c8-a955-aab940b83bf7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:04.639968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2fa94cb6-4a31-47c8-a955-aab940b83bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:04.773477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa94cb6-4a31-47c8-a955-aab940b83bf7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:04.831607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:04.882392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:04.883007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:04.883278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:04.883540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:04.884217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:04.886008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:40:04.886008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:40:04.886008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:40:04.887136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e3cf2-5eb8-4311-810e-8db66e5849a9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:04.910264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 741/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Fri Oct 28 09:40:04 2022] GET /identity/v3/projects => generated 2553 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:40:07.112508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff1f4e-1770-4392-9b23-47e2fa342d2e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:07.113100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff1f4e-1770-4392-9b23-47e2fa342d2e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:07.113210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff1f4e-1770-4392-9b23-47e2fa342d2e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:07.114526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 738/1479] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:07.120035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e3fc-e65d-4fb7-b66f-f09365da0a76 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:07.120455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e3fc-e65d-4fb7-b66f-f09365da0a76 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:07.120566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2785e3fc-e65d-4fb7-b66f-f09365da0a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:07.196939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2785e3fc-e65d-4fb7-b66f-f09365da0a76 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:07.198167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2785e3fc-e65d-4fb7-b66f-f09365da0a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:07.293834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 742/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:07.583690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff1f4e-1770-4392-9b23-47e2fa342d2e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:07.585334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-09ff1f4e-1770-4392-9b23-47e2fa342d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:07.595350 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:07.734456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff1f4e-1770-4392-9b23-47e2fa342d2e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:07.815116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:07.852111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:07.852644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:07.852917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:07.853185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:07.854892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:07.953992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:07.956181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d29a47-917d-4a74-8c53-f4bddf93bd12 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:07.976798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 739/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:40:10.313226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb07e9-6b42-42a0-b466-b87d34d5e088 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:10.313826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb07e9-6b42-42a0-b466-b87d34d5e088 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:10.313826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb07e9-6b42-42a0-b466-b87d34d5e088 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:10.315142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 743/1482] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:10.320547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe60577-505b-40ad-8564-b4f84b46abbb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:10.320999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe60577-505b-40ad-8564-b4f84b46abbb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:10.321113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe60577-505b-40ad-8564-b4f84b46abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:10.385237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8fe60577-505b-40ad-8564-b4f84b46abbb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:10.386661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fe60577-505b-40ad-8564-b4f84b46abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:10.434116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:10.490550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 740/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:10.772079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb07e9-6b42-42a0-b466-b87d34d5e088 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:10.773657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3bdb07e9-6b42-42a0-b466-b87d34d5e088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:10.798240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:10.887213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb07e9-6b42-42a0-b466-b87d34d5e088 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:10.970351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:10.971535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:10.971832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:10.972097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:10.973803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:11.057218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:11.093046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:11.094741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e480cc5-1d6c-4968-b65b-e8add9abf403 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:11.112617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 744/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:40:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:40:13.342461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8bab-4cb1-4367-93c2-423ac20d1791 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:13.343045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8bab-4cb1-4367-93c2-423ac20d1791 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:13.343045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d8bab-4cb1-4367-93c2-423ac20d1791 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:13.344664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 741/1485] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:13.350746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1621-cffc-4741-b4ba-fffc6cc4bf31 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:13.351157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1621-cffc-4741-b4ba-fffc6cc4bf31 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:13.351293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde1621-cffc-4741-b4ba-fffc6cc4bf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:13.417666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0dde1621-cffc-4741-b4ba-fffc6cc4bf31 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:13.418801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0dde1621-cffc-4741-b4ba-fffc6cc4bf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:13.525040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 745/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:13.815987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8bab-4cb1-4367-93c2-423ac20d1791 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:13.817559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce9d8bab-4cb1-4367-93c2-423ac20d1791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:13.928762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d8bab-4cb1-4367-93c2-423ac20d1791 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:14.005507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:14.007396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:14.008138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:14.008603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:14.009035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:14.010864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:14.143460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:14.145056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11da8b7-17ae-479f-a0cd-9ce644097a15 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:14.164988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 742/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:40:16.480044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbe452-19b5-4cd3-a06e-47f502b16f5c None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:16.480582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbe452-19b5-4cd3-a06e-47f502b16f5c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:16.480716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbe452-19b5-4cd3-a06e-47f502b16f5c None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:16.481995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 746/1488] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:16.487415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d5616-a826-4d5b-aec8-4f7f29d57c22 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:16.487859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d5616-a826-4d5b-aec8-4f7f29d57c22 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:16.487989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d5616-a826-4d5b-aec8-4f7f29d57c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:16.553548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d18d5616-a826-4d5b-aec8-4f7f29d57c22 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:16.554770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d18d5616-a826-4d5b-aec8-4f7f29d57c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:16.607568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:16.653431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 743/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:40:16.950995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbe452-19b5-4cd3-a06e-47f502b16f5c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:16.952590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1bbbe452-19b5-4cd3-a06e-47f502b16f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:16.989880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:17.070127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbe452-19b5-4cd3-a06e-47f502b16f5c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:17.153114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:17.153726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:17.153998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:17.154264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:17.155906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:17.240116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:17.268784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:17.270215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac19ad4-0ea5-4824-bdfe-2afd6613afe3 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:17.289282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 747/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:40:19.661710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6e4f6-1774-495c-b347-9fde919cadb3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:19.662257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6e4f6-1774-495c-b347-9fde919cadb3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:19.662257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6e4f6-1774-495c-b347-9fde919cadb3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:19.663578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 744/1491] 10.222.0.52 () {54 vars in 927 bytes} [Fri Oct 28 09:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:40:19.669321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06ce0-9cb3-4a9f-8200-2e734ba0d664 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:19.669782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06ce0-9cb3-4a9f-8200-2e734ba0d664 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:19.669902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc06ce0-9cb3-4a9f-8200-2e734ba0d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:19.753035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ecc06ce0-9cb3-4a9f-8200-2e734ba0d664 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:40:19.754268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ecc06ce0-9cb3-4a9f-8200-2e734ba0d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:19.854651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 748/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Fri Oct 28 09:40:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:40:20.145038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6e4f6-1774-495c-b347-9fde919cadb3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:40:20.146645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73d6e4f6-1774-495c-b347-9fde919cadb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:20.252479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6e4f6-1774-495c-b347-9fde919cadb3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:40:20.325965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:40:20.326569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:40:20.326848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:40:20.327117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:40:20.328747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:40:20.337469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:20.440198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:40:20.441662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad20a1-7998-4f67-b742-c77382c6be9a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:40:20.459692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 745/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:40:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:40:29.853086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:40:30.457404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=50 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:34.536885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:34.537619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:34.537619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:34.619579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:41:34.620835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:34.717293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 749/1494] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:41:34.725614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:34.726159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:34.726159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:34.803979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:41:34.805092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:34.901180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 746/1495] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:41:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:41:34.905924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:34.907718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:34.919101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26dc523-cfbc-4545-99ea-301a6ec1703b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:34.982299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:35.014913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:35.015521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.015826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.016122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.017309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:41:35.017604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:35.018674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:35.018674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:41:35.018674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:41:35.019559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:35.031803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 750/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:41:35.041970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:35.051280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db7037-be14-4332-a180-cc941f7c4331 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:35.112923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:35.147386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:35.148021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.148294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.148556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.149195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:35.150927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:35.150927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:41:35.150927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:41:35.152146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:35.208219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 747/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:41:35 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:41:35.212467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:35.214030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:35.249256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e319-1634-4f32-acdc-4dac32d896d3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:35.343089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:35.343690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.343991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.344265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.345423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:35.346675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:35.346675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:41:35.346675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:41:35.347907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:35.391809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 751/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:41:35 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:41:35.395792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:35.397431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:35.460068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab1df2-62e4-4f27-b7e2-4bd958afe9c8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:35.544530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:35.556238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:35.556853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.557127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.557484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.558040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:35.559378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:35.559378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:41:35.559378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:41:35.560015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:35.574801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 748/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:41:35.578916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:35.580659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:35.592258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa4f18-f16e-4a69-9b37-3b1ff9079f92 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:35.618031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:35.692045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:35.692722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.693029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.693340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.694090 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:35.695206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:35.695206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:41:35.695206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:41:35.696252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36979b-f0dc-4567-b3c6-ab807caa131c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:35.709532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 752/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:41:35.713712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:35.715257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:35.749542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb902b9-62c2-40ec-b9b1-4b3e002bf2a4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:35.844641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:35.845290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.845820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.846085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] PATH_INFO: `/v3/projects/aef3b2fc4d6e440294eaca4fda9a159b/users/b23635d667f24e3e982b052356a9133d/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.867926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef3b2fc4d6e440294eaca4fda9a159b, user_id=b23635d667f24e3e982b052356a9133d, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:35.868952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:35.868952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:41:35.868952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:41:35.870893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa123ed-b18f-4c2c-b274-1b7211ca168d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:35.897006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 749/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:41:35 2022] PUT /identity/v3/projects/aef3b2fc4d6e440294eaca4fda9a159b/users/b23635d667f24e3e982b052356a9133d/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:41:35.908107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21610314-06c8-4c85-a0f0-134b0ccbe02b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:35.908411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21610314-06c8-4c85-a0f0-134b0ccbe02b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:35.908683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21610314-06c8-4c85-a0f0-134b0ccbe02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:35.970793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-21610314-06c8-4c85-a0f0-134b0ccbe02b None None] MFA Rules not processed for user `b23635d667f24e3e982b052356a9133d`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:41:35.972195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-21610314-06c8-4c85-a0f0-134b0ccbe02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:36.050691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 753/1502] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:41:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:41:36.064525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71916-a8f3-4d1e-9c88-f2c1f7e23079 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:36.064820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71916-a8f3-4d1e-9c88-f2c1f7e23079 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:36.065087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd71916-a8f3-4d1e-9c88-f2c1f7e23079 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:36.066410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 750/1503] 10.222.0.52 () {54 vars in 957 bytes} [Fri Oct 28 09:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:41:36.071299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9997-a3b4-4482-8e69-b40fdb8010f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:36.071579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9997-a3b4-4482-8e69-b40fdb8010f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:36.071904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9997-a3b4-4482-8e69-b40fdb8010f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:36.105512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:36.152372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b0fd9997-a3b4-4482-8e69-b40fdb8010f5 None None] MFA Rules not processed for user `16332ba5fe89453292d21325b333407a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:41:36.153561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b0fd9997-a3b4-4482-8e69-b40fdb8010f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:36.229478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 754/1504] 10.222.0.52 () {58 vars in 1054 bytes} [Fri Oct 28 09:41:36 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:41:36.239223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71916-a8f3-4d1e-9c88-f2c1f7e23079 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:36.240803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3bd71916-a8f3-4d1e-9c88-f2c1f7e23079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:36.291728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:36.347209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd71916-a8f3-4d1e-9c88-f2c1f7e23079 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:36.421096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:36.421709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:36.421981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:36.422249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:36.424013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:36.522288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:36.523637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:36.527061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:36.542718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 751/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:41:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:41:36.588405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe831a-63cf-4c64-b379-9839b2f8eaca None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:36.588846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe831a-63cf-4c64-b379-9839b2f8eaca None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:36.588957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fe831a-63cf-4c64-b379-9839b2f8eaca None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:36.590187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 755/1506] 10.222.0.52 () {52 vars in 830 bytes} [Fri Oct 28 09:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:41:36.593879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:36.596055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:36.696295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be241-141b-4b5b-8944-a6eeb810d89d service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:36.793492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:36.794154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:36.794304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:36.794573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] PATH_INFO: `/v3/users/b23635d667f24e3e982b052356a9133d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:36.808234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23635d667f24e3e982b052356a9133d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:36.809275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:36.809275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:41:36.809275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:41:36.810484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:36.820473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:36.876673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23635d667f24e3e982b052356a9133d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:36.885939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23635d667f24e3e982b052356a9133d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:36.909784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23635d667f24e3e982b052356a9133d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:36.910241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23635d667f24e3e982b052356a9133d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:36.924658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23635d667f24e3e982b052356a9133d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:36.930558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 752/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:41:36 2022] DELETE /identity/v3/users/b23635d667f24e3e982b052356a9133d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:41:36.934969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:36.935257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:36.935557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:36.994267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:41:36.995473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:37.072266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:37.086397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 756/1508] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:41:37.097625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:37.098805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:37.202515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cddd7-fd79-495a-a543-ca097e4b3b68 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:37.259626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:37.281143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:37.281760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:37.282569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:37.282837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:37.284553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:37.394887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:37.396301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea675e70-f9cf-485c-9044-959b5a67f59b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:37.415075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 753/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:41:37.859779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:41:37.861610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:41:37.875265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21f1cc-7407-41d8-abdf-50a7f62a9cd9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:41:37.978948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:41:37.979677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:41:37.979975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:41:37.980236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] PATH_INFO: `/v3/projects/aef3b2fc4d6e440294eaca4fda9a159b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:41:37.991720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef3b2fc4d6e440294eaca4fda9a159b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:41:37.992761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:41:37.992761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:41:37.992761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:41:37.993958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:41:38.053458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef3b2fc4d6e440294eaca4fda9a159b', 'request_id': 'req-e3565748-4ca2-4fa2-b6ba-15199e0ab998'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:38.059502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef3b2fc4d6e440294eaca4fda9a159b', 'request_id': 'req-e3565748-4ca2-4fa2-b6ba-15199e0ab998'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:38.072845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef3b2fc4d6e440294eaca4fda9a159b', 'request_id': 'req-e3565748-4ca2-4fa2-b6ba-15199e0ab998'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:38.099403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] The token cache is being invalidate because project aef3b2fc4d6e440294eaca4fda9a159b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:41:38.099691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:38.100145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e3565748-4ca2-4fa2-b6ba-15199e0ab998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:41:38.101940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 757/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:41:37 2022] DELETE /identity/v3/projects/aef3b2fc4d6e440294eaca4fda9a159b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:41:47.412882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=62 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:41:48.097446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:13.301187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b69575-7bf2-4c81-aaf9-cba09cdf34cf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:13.301799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b69575-7bf2-4c81-aaf9-cba09cdf34cf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:13.301799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b69575-7bf2-4c81-aaf9-cba09cdf34cf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:13.303069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:13.307925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ee5eb-4e41-471e-815c-6b813411d617 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:13.308374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ee5eb-4e41-471e-815c-6b813411d617 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:13.308479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ee5eb-4e41-471e-815c-6b813411d617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:13.387262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-374ee5eb-4e41-471e-815c-6b813411d617 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:13.388513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-374ee5eb-4e41-471e-815c-6b813411d617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:13.491398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:13.643444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b69575-7bf2-4c81-aaf9-cba09cdf34cf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:13.645580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-89b69575-7bf2-4c81-aaf9-cba09cdf34cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:13.761391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b69575-7bf2-4c81-aaf9-cba09cdf34cf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:13.766903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c31fb-5307-4ea7-90d3-866325ed6bbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:13.769639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c31fb-5307-4ea7-90d3-866325ed6bbb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:13.769933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c31fb-5307-4ea7-90d3-866325ed6bbb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:13.770206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c31fb-5307-4ea7-90d3-866325ed6bbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:13.778549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c31fb-5307-4ea7-90d3-866325ed6bbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:13.780170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c31fb-5307-4ea7-90d3-866325ed6bbb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:13.790048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 755/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:14.608119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d26178-2a2b-4c14-9888-49e0098b0df6 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:14.608657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d26178-2a2b-4c14-9888-49e0098b0df6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:14.608779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d26178-2a2b-4c14-9888-49e0098b0df6 None None] PATH_INFO: `/v3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:14.610179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 759/1514] 10.222.0.58 () {54 vars in 925 bytes} [Fri Oct 28 09:43:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:43:14.617032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7153-52b9-4191-9c86-94527bd2bbd3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:14.617486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7153-52b9-4191-9c86-94527bd2bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:14.617699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d7153-52b9-4191-9c86-94527bd2bbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:14.677377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-407d7153-52b9-4191-9c86-94527bd2bbd3 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:14.678559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-407d7153-52b9-4191-9c86-94527bd2bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:14.745787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:14.751430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 756/1515] 10.222.0.58 () {58 vars in 1015 bytes} [Fri Oct 28 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:14.875802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbffecd-8209-4a0b-a340-15f156660922 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:14.876088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbffecd-8209-4a0b-a340-15f156660922 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:14.876363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbffecd-8209-4a0b-a340-15f156660922 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:14.878261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:14.884581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f24a8-60bd-458a-9b61-bf84808dc0ea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:14.884866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f24a8-60bd-458a-9b61-bf84808dc0ea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:14.885139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f24a8-60bd-458a-9b61-bf84808dc0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:14.905716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-231f24a8-60bd-458a-9b61-bf84808dc0ea None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:14.906901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-231f24a8-60bd-458a-9b61-bf84808dc0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:14.914781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:15.063707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbffecd-8209-4a0b-a340-15f156660922 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:15.078633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbffecd-8209-4a0b-a340-15f156660922 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:15.084692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8556e-1696-42a4-a66c-98151de2ff99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:15.085294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8556e-1696-42a4-a66c-98151de2ff99 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:15.085571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8556e-1696-42a4-a66c-98151de2ff99 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:15.085842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8556e-1696-42a4-a66c-98151de2ff99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:15.093974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8556e-1696-42a4-a66c-98151de2ff99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:15.095563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8556e-1696-42a4-a66c-98151de2ff99 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:15.105130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 761/1518] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:23.429983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc74b4e-7dda-487b-b484-a1b01508caa8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:23.430424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc74b4e-7dda-487b-b484-a1b01508caa8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:23.430521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc74b4e-7dda-487b-b484-a1b01508caa8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:23.431879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 758/1519] 10.222.0.52 () {54 vars in 961 bytes} [Fri Oct 28 09:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:23.436966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51ddb2-9e2d-4894-b838-6077bcd3ce1f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:23.437434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51ddb2-9e2d-4894-b838-6077bcd3ce1f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:23.437553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51ddb2-9e2d-4894-b838-6077bcd3ce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:23.457717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dd51ddb2-9e2d-4894-b838-6077bcd3ce1f None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:23.458907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd51ddb2-9e2d-4894-b838-6077bcd3ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:23.467388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 762/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Fri Oct 28 09:43:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:23.475156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc74b4e-7dda-487b-b484-a1b01508caa8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:23.488783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc74b4e-7dda-487b-b484-a1b01508caa8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:23.493672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283d98c-f583-4207-90e4-d7d7b5a9d57d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:23.494259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d98c-f583-4207-90e4-d7d7b5a9d57d service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:23.494535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d98c-f583-4207-90e4-d7d7b5a9d57d service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:23.494805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283d98c-f583-4207-90e4-d7d7b5a9d57d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:23.503452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283d98c-f583-4207-90e4-d7d7b5a9d57d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:23.505075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283d98c-f583-4207-90e4-d7d7b5a9d57d service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:23.514520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 759/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:26.574861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb935d-e829-4750-b426-36b025b2c0e9 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:26.575430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb935d-e829-4750-b426-36b025b2c0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:26.575430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb935d-e829-4750-b426-36b025b2c0e9 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:26.576809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:26.583581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3003-1f20-48ed-ae7d-2fe01d2c36e9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:26.584072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3003-1f20-48ed-ae7d-2fe01d2c36e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:26.584191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df3003-1f20-48ed-ae7d-2fe01d2c36e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:26.604038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-73df3003-1f20-48ed-ae7d-2fe01d2c36e9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:26.605233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73df3003-1f20-48ed-ae7d-2fe01d2c36e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:26.612981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:26.767764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb935d-e829-4750-b426-36b025b2c0e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:26.782746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb935d-e829-4750-b426-36b025b2c0e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:26.788068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beab451-4f2a-489d-8fae-94fdf314465b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:26.788653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab451-4f2a-489d-8fae-94fdf314465b service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:26.788924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab451-4f2a-489d-8fae-94fdf314465b service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:26.789192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beab451-4f2a-489d-8fae-94fdf314465b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:26.796994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beab451-4f2a-489d-8fae-94fdf314465b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:26.798553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beab451-4f2a-489d-8fae-94fdf314465b service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:26.807622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 764/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:27.893551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c3f7-9a47-4990-bce0-302f9c967d47 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:27.894161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c3f7-9a47-4990-bce0-302f9c967d47 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:27.894161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c3f7-9a47-4990-bce0-302f9c967d47 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:27.895432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 28 09:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:27.901833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a14ad-dad7-4ce7-a722-ffe8190aa27f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:27.902266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a14ad-dad7-4ce7-a722-ffe8190aa27f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:27.902371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a14ad-dad7-4ce7-a722-ffe8190aa27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:27.921556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-552a14ad-dad7-4ce7-a722-ffe8190aa27f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:27.922763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-552a14ad-dad7-4ce7-a722-ffe8190aa27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:27.931238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 28 09:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:28.083115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344c3f7-9a47-4990-bce0-302f9c967d47 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:28.097246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344c3f7-9a47-4990-bce0-302f9c967d47 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:28.103015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87219698-a88c-4832-ba09-66e4b49a16fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:28.103631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219698-a88c-4832-ba09-66e4b49a16fa service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:28.103938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219698-a88c-4832-ba09-66e4b49a16fa service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:28.104215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87219698-a88c-4832-ba09-66e4b49a16fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:28.113465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87219698-a88c-4832-ba09-66e4b49a16fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:28.115071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87219698-a88c-4832-ba09-66e4b49a16fa service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:28.125026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 762/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:32.536687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a0e5a-23cd-4d0c-856d-ae255b778248 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:32.537252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a0e5a-23cd-4d0c-856d-ae255b778248 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:32.537252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a0e5a-23cd-4d0c-856d-ae255b778248 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:32.538571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:32.544524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1e496-1ff5-486d-a0ec-613778855aee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:32.545037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1e496-1ff5-486d-a0ec-613778855aee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:32.545136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1e496-1ff5-486d-a0ec-613778855aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:32.565952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57a1e496-1ff5-486d-a0ec-613778855aee None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:32.567310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57a1e496-1ff5-486d-a0ec-613778855aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:32.575378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:32.863034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a0e5a-23cd-4d0c-856d-ae255b778248 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:32.864856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6a6a0e5a-23cd-4d0c-856d-ae255b778248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:32.899147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:32.946666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a0e5a-23cd-4d0c-856d-ae255b778248 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:32.950311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7587c8d8-84e6-4c77-b050-32c9953d1d68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:32.950917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587c8d8-84e6-4c77-b050-32c9953d1d68 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:32.951197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587c8d8-84e6-4c77-b050-32c9953d1d68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:32.951464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7587c8d8-84e6-4c77-b050-32c9953d1d68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:32.960754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587c8d8-84e6-4c77-b050-32c9953d1d68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:32.962179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7587c8d8-84e6-4c77-b050-32c9953d1d68 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:32.972374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 767/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:35.144127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bddd1-a3d7-4aa1-b528-36226c45f8a5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:35.144690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bddd1-a3d7-4aa1-b528-36226c45f8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:35.144690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0bddd1-a3d7-4aa1-b528-36226c45f8a5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:35.146486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:35.151299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31aa5c-9310-41db-83f7-37ba6d1013fa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:35.151687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31aa5c-9310-41db-83f7-37ba6d1013fa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:35.151893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31aa5c-9310-41db-83f7-37ba6d1013fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:35.171936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ce31aa5c-9310-41db-83f7-37ba6d1013fa None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:35.173131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ce31aa5c-9310-41db-83f7-37ba6d1013fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:35.181599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:35.466555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bddd1-a3d7-4aa1-b528-36226c45f8a5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:35.477730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0bddd1-a3d7-4aa1-b528-36226c45f8a5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:35.481275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c3f0e-8e27-442e-87ed-248fd196cd21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:35.481877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3f0e-8e27-442e-87ed-248fd196cd21 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:35.482154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3f0e-8e27-442e-87ed-248fd196cd21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:35.482424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c3f0e-8e27-442e-87ed-248fd196cd21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:35.493189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c3f0e-8e27-442e-87ed-248fd196cd21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:35.494586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c3f0e-8e27-442e-87ed-248fd196cd21 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:35.505076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 765/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:39.199626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea196a3-ab99-4efc-8113-917514ae256d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:39.200208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea196a3-ab99-4efc-8113-917514ae256d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:39.200208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea196a3-ab99-4efc-8113-917514ae256d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:39.201577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:39.207229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a40d-a987-4093-ba3d-65aee52bca87 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:39.207665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a40d-a987-4093-ba3d-65aee52bca87 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:39.207790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838a40d-a987-4093-ba3d-65aee52bca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:39.227615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2838a40d-a987-4093-ba3d-65aee52bca87 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:39.228873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2838a40d-a987-4093-ba3d-65aee52bca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:39.236836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:39.537868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea196a3-ab99-4efc-8113-917514ae256d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:39.539508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9ea196a3-ab99-4efc-8113-917514ae256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:39.617115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea196a3-ab99-4efc-8113-917514ae256d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:39.620672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609024f-c37b-4965-9f24-89dbcb632064 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:39.621273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609024f-c37b-4965-9f24-89dbcb632064 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:39.621547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609024f-c37b-4965-9f24-89dbcb632064 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:39.621815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609024f-c37b-4965-9f24-89dbcb632064 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:39.630635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609024f-c37b-4965-9f24-89dbcb632064 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:39.632086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609024f-c37b-4965-9f24-89dbcb632064 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:39.641338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 770/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:41.774511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd519-73da-4bbb-ba66-3b249df7139d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:41.774984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd519-73da-4bbb-ba66-3b249df7139d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:41.775110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477dd519-73da-4bbb-ba66-3b249df7139d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:41.776481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 28 09:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:43:41.781731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b642554-6ee4-42fa-a50f-3ff7693b29de None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:41.782149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b642554-6ee4-42fa-a50f-3ff7693b29de None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:41.782277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b642554-6ee4-42fa-a50f-3ff7693b29de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:41.803044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7b642554-6ee4-42fa-a50f-3ff7693b29de None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:43:41.804285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7b642554-6ee4-42fa-a50f-3ff7693b29de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:41.812077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 28 09:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:43:42.106990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd519-73da-4bbb-ba66-3b249df7139d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:43:42.108707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-477dd519-73da-4bbb-ba66-3b249df7139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:43:42.122139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477dd519-73da-4bbb-ba66-3b249df7139d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:43:42.126020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f04d84-1dfa-4894-9e60-8ac7252c16ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:43:42.126656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f04d84-1dfa-4894-9e60-8ac7252c16ce service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:43:42.126929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f04d84-1dfa-4894-9e60-8ac7252c16ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:43:42.127200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f04d84-1dfa-4894-9e60-8ac7252c16ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:43:42.137446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f04d84-1dfa-4894-9e60-8ac7252c16ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:43:42.138892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f04d84-1dfa-4894-9e60-8ac7252c16ce service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:43:42.149428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 768/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:43:51.797732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=63 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:43:52.144986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=37 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:17.734342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:17.735978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:17.736454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:17.762678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:17.764069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:17.777520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 772/1540] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:17.787486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:17.788755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:17.788755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:17.820905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:17.822126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:17.833578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 769/1541] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:17.838890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:17.852032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ea169-bb1d-4742-8539-cbab6ec89ad0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:17.860087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:17.860724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:17.860999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:17.861265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:17.863327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:17.863675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:17.865351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:17.881476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 773/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:17.890076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:17.901974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd20feb-9c53-4f7b-b288-da28ed0a3f12 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:17.912556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:17.913228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:17.913503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:17.913768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:17.914416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:17.916399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:17.980650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 770/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:17 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:17.984627 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.001973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a6b24-3397-4914-815f-c1a1861d35ec admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.099645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.100317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.100647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.101999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.102924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.104958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.145664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 774/1544] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:17 2022] POST /identity/v3/users => generated 283 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:18.151115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.170440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2028b86b-12e3-45cf-90d8-90c2de0f5d65 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.181032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.181671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.181946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.182238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.182865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.184513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.199936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 771/1545] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:18.204537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.221161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58ba11-fa77-4c12-90a8-83d2e3241fae admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.234921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2963c-9d26-4e59-ba7b-0a445f2e4c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.235471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2963c-9d26-4e59-ba7b-0a445f2e4c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.235766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2963c-9d26-4e59-ba7b-0a445f2e4c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.236043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2963c-9d26-4e59-ba7b-0a445f2e4c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.237108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2963c-9d26-4e59-ba7b-0a445f2e4c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.239022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2963c-9d26-4e59-ba7b-0a445f2e4c96 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.257684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 775/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:18.262437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.273179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e83e94-9cf5-4723-8e74-00c41b556eeb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.289538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f9121-e6d9-45d4-b3da-1bea92a4420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.290240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9121-e6d9-45d4-b3da-1bea92a4420e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.290520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9121-e6d9-45d4-b3da-1bea92a4420e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.290797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f9121-e6d9-45d4-b3da-1bea92a4420e admin admin] PATH_INFO: `/v3/projects/28039617399a4a5094039022d287ccd6/users/52a7bc3f048c47f6b3d8616a2b9bd3f2/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.327788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f9121-e6d9-45d4-b3da-1bea92a4420e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28039617399a4a5094039022d287ccd6, user_id=52a7bc3f048c47f6b3d8616a2b9bd3f2, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.330664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f9121-e6d9-45d4-b3da-1bea92a4420e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.361650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 772/1547] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:18 2022] PUT /identity/v3/projects/28039617399a4a5094039022d287ccd6/users/52a7bc3f048c47f6b3d8616a2b9bd3f2/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:18.365124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250de0-0e9e-4bc0-87bc-5ca30ef2a780 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.365455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250de0-0e9e-4bc0-87bc-5ca30ef2a780 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.365722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250de0-0e9e-4bc0-87bc-5ca30ef2a780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.375140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c34ae-8d9c-438b-8e7b-27bce05ac75d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.375500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c34ae-8d9c-438b-8e7b-27bce05ac75d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.375683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c34ae-8d9c-438b-8e7b-27bce05ac75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.398787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-da250de0-0e9e-4bc0-87bc-5ca30ef2a780 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.401016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-da250de0-0e9e-4bc0-87bc-5ca30ef2a780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.434863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e35c34ae-8d9c-438b-8e7b-27bce05ac75d None None] MFA Rules not processed for user `52a7bc3f048c47f6b3d8616a2b9bd3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.436373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e35c34ae-8d9c-438b-8e7b-27bce05ac75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.537183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 776/1548] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.540982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76282-2967-4a02-9908-dcc0ae6c4478 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.541754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76282-2967-4a02-9908-dcc0ae6c4478 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.542419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76282-2967-4a02-9908-dcc0ae6c4478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.569621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 773/1549] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 28 09:45:18.571100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.571379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.571649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.609174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.610432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.612839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-16c76282-2967-4a02-9908-dcc0ae6c4478 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.614048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-16c76282-2967-4a02-9908-dcc0ae6c4478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.625597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 774/1550] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.626456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.632670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 777/1551] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.634047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.634363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.634657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.678224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.680511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.709557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 778/1552] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.710489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.714575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336a382-3e57-48ac-87c4-0fb64c8545f9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.720999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafef43d-c062-4258-942f-80ecdd81a34f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.721366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62aac7-5408-48c6-b28a-67a1bb3a2549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.722473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62aac7-5408-48c6-b28a-67a1bb3a2549 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.723050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62aac7-5408-48c6-b28a-67a1bb3a2549 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.723487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62aac7-5408-48c6-b28a-67a1bb3a2549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.726777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.727509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.728028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.728412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.729363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:18.729796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.731633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124cbb-0d52-46c2-a3f6-8f56716fc00a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.738062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62aac7-5408-48c6-b28a-67a1bb3a2549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.738062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62aac7-5408-48c6-b28a-67a1bb3a2549 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.750776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 775/1553] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:18.752214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fca43-a241-4bd0-9bf1-a98e62154de3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.752710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 779/1554] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:18.753218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fca43-a241-4bd0-9bf1-a98e62154de3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.753218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fca43-a241-4bd0-9bf1-a98e62154de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.753864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f615-8395-4350-b47c-41e1e05774ac None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.754143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f615-8395-4350-b47c-41e1e05774ac None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.754413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f615-8395-4350-b47c-41e1e05774ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.777899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-045fca43-a241-4bd0-9bf1-a98e62154de3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.779094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-045fca43-a241-4bd0-9bf1-a98e62154de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.779832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2d61f615-8395-4350-b47c-41e1e05774ac None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.781088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2d61f615-8395-4350-b47c-41e1e05774ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.787065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 776/1555] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.789203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c364d-074b-4e19-877e-e5a212802acf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.792141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c364d-074b-4e19-877e-e5a212802acf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.792254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 780/1556] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.792371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c364d-074b-4e19-877e-e5a212802acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.792481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a701fa-a956-459a-b401-6d85009019e2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.792481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a701fa-a956-459a-b401-6d85009019e2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.792481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a701fa-a956-459a-b401-6d85009019e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.812385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:18.813422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-06a701fa-a956-459a-b401-6d85009019e2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.814370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06a701fa-a956-459a-b401-6d85009019e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.818671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2c0c364d-074b-4e19-877e-e5a212802acf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.821750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c0c364d-074b-4e19-877e-e5a212802acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.824911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 781/1557] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.826150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.826434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.826703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.831103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 777/1558] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.832657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.833147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.833737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.851866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.853259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.860741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.861970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:18.864108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 782/1559] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.864958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.875836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18037298-a379-4d54-81f4-b80c84b84562 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.876117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 778/1560] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:18.876959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.880837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.881463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.882573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.882843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.883491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.884546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:45:18.884546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:45:18.884546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:45:18.885799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.889791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423342b-fa54-4a84-8993-e8b395a3f8b2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:18.895731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:18.896516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.896902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.897272 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.898093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:18.898455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:18.899567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:45:18.899567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:45:18.899567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:45:18.899904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:18.900753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9a67-2121-4d4a-a8c1-cb1b9be46d5b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:18.914636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 779/1561] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:18.916185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ca634-a971-4753-bf58-72a6f5806755 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:18.916466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ca634-a971-4753-bf58-72a6f5806755 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:18.916749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ca634-a971-4753-bf58-72a6f5806755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:18.924536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 783/1562] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:18.925114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:18.939445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-666ca634-a971-4753-bf58-72a6f5806755 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:18.940689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-666ca634-a971-4753-bf58-72a6f5806755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:19.000984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ec9b-bbbe-49db-b4d6-ef7acaa3a062 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.009695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.011969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 780/1563] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:19.012892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.013342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.014781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed1eb1-68b3-457d-9647-2fcd402468d0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.015418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.016321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:19.016706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed1eb1-68b3-457d-9647-2fcd402468d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.017011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.017128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed1eb1-68b3-457d-9647-2fcd402468d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.018966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfface1-ca73-475c-bca8-074c63c0a64c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.036692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 784/1564] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:19.038674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.038967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.039286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.045343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-05ed1eb1-68b3-457d-9647-2fcd402468d0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:19.046856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-05ed1eb1-68b3-457d-9647-2fcd402468d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:19.058592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 781/1565] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:19.060443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.060729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.061001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.072736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:19.074070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:19.082187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:19.083436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:19.086796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 785/1566] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:19.087707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.096407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 782/1567] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:19.096407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.102376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bcc53-004b-4750-8d42-0ea2a05691ec None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.106348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185b0c9-99ab-49a7-ba8f-e54723e8501e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.109628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.110541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.111018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.111449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.111965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.112580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.112778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.112900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.113148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.113733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.114888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:45:19.114888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:45:19.114888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:45:19.115249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.116113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.144477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 783/1568] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 291 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.145560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.151553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 786/1569] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.152607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.156287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f2f66-a041-4913-ba2a-5d4b0dc890b5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.221182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d60c86-6c56-4784-a8bc-ec7fda55fdb9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.222621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.223243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.223523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.223842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.224460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:19.224761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.227266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.227733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.227994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.228381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.228655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.229287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.231182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.247175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 784/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:19.248296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.260211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fc763-bb68-4145-8d6d-f77613e755a9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.265194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.265831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.266201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.266479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.267133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:19.267452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.269155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.281544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 787/1571] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.282378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.292672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 785/1572] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:19.294788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.313238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda0334a-b012-47bb-9545-26b7b75f47f8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.349182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:19.366939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:19.372691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eaed33-af1c-4d3e-b961-ce5b170b6109 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.379441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.380235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.380502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.380780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.381502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:19.382033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.383729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.398006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 788/1573] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:19.399191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.414890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729398a-9330-4802-bfbb-68ec95a98ce4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.416273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.416867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.417139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.417406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.418053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:19.418348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.420033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.424513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.425415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.426050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.426659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.427602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.430018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.432924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 786/1574] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:19.442077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.443652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 789/1575] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:19.453497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742ebde-27fb-4548-b1c1-29111abd7d73 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.456193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.463365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.464031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.464305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.464570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.465202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.468499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.470952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef91830-502d-41d4-91e6-2c508560bafd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.476912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.477955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.478334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.478915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.479641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.482204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.494761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 787/1576] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 286 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.495782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.507086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da941e98-0c79-4bd6-8113-56e0a96bd79b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.512488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.513201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.513555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.513894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.514643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.518108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 790/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.518108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.518370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.542480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 788/1578] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.543430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.554142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3c388-d630-4b0c-98d0-51d40a2557ef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.592050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfd52-f6ef-4de1-a920-9ffb8914b32d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.596887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.597452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.597729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.597990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.598621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.600591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.613405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.614110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.614390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.614656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.615285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.617285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.628867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 791/1579] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.629692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.647052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 789/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.648782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.696729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f213b35-ee3e-4e5b-a7cd-5b7778d3ce27 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.718792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eec3d-7d4d-4248-a5b8-843053245eae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.723412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.724058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.724331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.724602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.725246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.727704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.731191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:19.738781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 790/1581] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:19.739657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.749764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e4dff-624e-4b80-81c0-73fa18dad99f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.754731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.755337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.755610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.755905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.756567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.758222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.762137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.762779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.763051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.763347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.763826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:19.764008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.765892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.771880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 791/1582] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:19.773347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.784015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b7913-55ff-4957-bec6-91470b58d4e1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.789396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.790031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.790312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.790575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.791200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.793182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.806377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 792/1583] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/projects => generated 375 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.807175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.817426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 792/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.818300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.830206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a741cf-117b-46b0-a742-f1f4114b88e3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.877976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4576e76-dec4-4f60-bf7e-c0a4c074c937 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.883325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.883990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.884250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.884562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.885141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.886732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.896286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.897021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 793/1585] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:19.897274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.898318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.898595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.898780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.899899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.903365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.912579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd4299-7a89-4dde-adae-0569ce52ea71 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.917438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.918062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.918335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.918606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.919243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.921199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.930147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 793/1586] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.930147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.940435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14cc1-419f-464c-b89d-68dea54991f9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.946991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 794/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:19.946991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.948054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.948729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.949004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.949274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] PATH_INFO: `/v3/projects/10eecff17fe242a7b93f7d5e88aee49d/users/294d317eec794729a7ffd85d42288b21/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.956654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985220e-5008-4319-b064-28dddc6330bd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:19.962845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:19.963453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:19.963724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:19.964020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:19.964750 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.965921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10eecff17fe242a7b93f7d5e88aee49d, user_id=294d317eec794729a7ffd85d42288b21, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:19.966358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.968653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:19.976772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 795/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:19.977505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.984406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 794/1589] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:19 2022] PUT /identity/v3/projects/10eecff17fe242a7b93f7d5e88aee49d/users/294d317eec794729a7ffd85d42288b21/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:19.985515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:19.987193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731d34d-895e-47b7-b96b-677168058c1f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.046033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.046645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.046923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.047192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.047727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c581428-79cb-46d1-b2b1-c7bd4fc0aaf2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.047883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.049789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.057838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.058396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.058675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.058947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.059628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.061287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.073335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 795/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.078255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 796/1591] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:19 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:20.078255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.078518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.087010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ccf5b-8b08-4cb3-967a-96d3a8f8017f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.087690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581be9a-882d-485a-8048-ec60c2c17478 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.091732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.092300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.092431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.092619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.092888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.092989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.093160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.093428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.093523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.094063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.095089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.095643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.108720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 797/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.108936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 796/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.109047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.109145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.117612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726788d-b32a-4b7e-a8dd-453e538e2de3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.118384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d37a8d-72be-4ca9-99d6-d8bac396eac1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.122120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.122769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.122892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5935d-b6e3-4255-9b4a-b571459c7a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.123044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.123324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] PATH_INFO: `/v3/projects/7bf643c5c7aa47468b2796ef67ad5321/users/f06db17158464566aae64758ef09edba/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.123444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5935d-b6e3-4255-9b4a-b571459c7a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.123704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5935d-b6e3-4255-9b4a-b571459c7a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.124008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5935d-b6e3-4255-9b4a-b571459c7a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.124643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5935d-b6e3-4255-9b4a-b571459c7a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.126222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5935d-b6e3-4255-9b4a-b571459c7a2c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.127719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:20.141397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf643c5c7aa47468b2796ef67ad5321, user_id=f06db17158464566aae64758ef09edba, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.144001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 798/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.144659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.145362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.145693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.145912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.159590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 797/1595] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:20 2022] PUT /identity/v3/projects/7bf643c5c7aa47468b2796ef67ad5321/users/f06db17158464566aae64758ef09edba/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:20.160478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.170072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:20.182618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] MFA Rules not processed for user `294d317eec794729a7ffd85d42288b21`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:20.183871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:20.224399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febdacd-9553-4b55-ae76-37b0d7389199 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.229455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.230225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.230505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.230772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.231413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.233067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.243690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 798/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.244558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.253568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 799/1597] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:20.254410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2b295-a4f9-4913-9181-a498cf03bf7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.254544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.261535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.262180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.262463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.262726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.263369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.265065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.266795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea6920-d890-41f4-98dd-9ffa0b9b0a03 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.271372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.272061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.272335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.272612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] PATH_INFO: `/v3/projects/981def9b53d4448abde1baf030e5c63c/users/5e01f518dbd947bfafa6aecdccd9cfd5/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.278941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 799/1598] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.278941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.287325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e97e7-8361-4987-95fd-ef30124c82dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.289253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981def9b53d4448abde1baf030e5c63c, user_id=5e01f518dbd947bfafa6aecdccd9cfd5, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.290332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:45:20.290332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:45:20.290332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:45:20.292575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.292708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f0789-f063-483c-a06a-fe5818d4be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.293237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0789-f063-483c-a06a-fe5818d4be4e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.293511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0789-f063-483c-a06a-fe5818d4be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.293775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f0789-f063-483c-a06a-fe5818d4be4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.294409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f0789-f063-483c-a06a-fe5818d4be4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.296608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f0789-f063-483c-a06a-fe5818d4be4e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.306891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 800/1599] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:20 2022] PUT /identity/v3/projects/981def9b53d4448abde1baf030e5c63c/users/5e01f518dbd947bfafa6aecdccd9cfd5/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:20.307727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.315201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 800/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.315201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.315201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.315201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.345244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] MFA Rules not processed for user `f06db17158464566aae64758ef09edba`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:20.346592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:20.390965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ce398-e5f5-4af4-84ab-b8711928cc34 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.398362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.399223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.399645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.400131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.401207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.403900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.415640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 801/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.416667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.428221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f20c9-0540-418d-a465-0d3300370946 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.432416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 801/1602] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:20.433266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.434811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.435952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.436557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.436913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] PATH_INFO: `/v3/projects/445369342220483b9a18e7cc39ae5651/users/88f1886cc5cc40ecb284b7a366ef1f7e/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.449438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c102d2-67f0-4636-b93d-4aad3b60dd93 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.454466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5c512-2b29-4a4e-8db0-26bd3a240792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.454640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445369342220483b9a18e7cc39ae5651, user_id=88f1886cc5cc40ecb284b7a366ef1f7e, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.455087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5c512-2b29-4a4e-8db0-26bd3a240792 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.455370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5c512-2b29-4a4e-8db0-26bd3a240792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.455732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5c512-2b29-4a4e-8db0-26bd3a240792 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.456303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5c512-2b29-4a4e-8db0-26bd3a240792 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.457980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.458322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5c512-2b29-4a4e-8db0-26bd3a240792 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.490549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 802/1603] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:20 2022] PUT /identity/v3/projects/445369342220483b9a18e7cc39ae5651/users/88f1886cc5cc40ecb284b7a366ef1f7e/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:20.492302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.502455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 802/1604] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:20.503733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.504048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.504321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.525639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:20.526852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:20.540742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:20.578106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354ac37-7be5-4f38-b217-b27733084a95 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.586072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:20.607485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 803/1605] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:20.608660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.620324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ad57d-9bcc-475e-954c-450835f75a7f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.625334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183033ca-29f5-4f3b-a4c0-4eb0bead14b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.626134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183033ca-29f5-4f3b-a4c0-4eb0bead14b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.626507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183033ca-29f5-4f3b-a4c0-4eb0bead14b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.626872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183033ca-29f5-4f3b-a4c0-4eb0bead14b2 admin admin] PATH_INFO: `/v3/projects/0f6964429ff54bb0875a37c8d323562d/users/032b078fb1b1451ea594f6ecaf2fd366/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.643957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183033ca-29f5-4f3b-a4c0-4eb0bead14b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6964429ff54bb0875a37c8d323562d, user_id=032b078fb1b1451ea594f6ecaf2fd366, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.644844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.645449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.645721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.645987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.646620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.647402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183033ca-29f5-4f3b-a4c0-4eb0bead14b2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.648294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.659513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 803/1606] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:20.660318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.665222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 804/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:20 2022] PUT /identity/v3/projects/0f6964429ff54bb0875a37c8d323562d/users/032b078fb1b1451ea594f6ecaf2fd366/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:20.666441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54309f-ea16-482f-9f82-bf293a9067b5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.666823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54309f-ea16-482f-9f82-bf293a9067b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.667224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54309f-ea16-482f-9f82-bf293a9067b5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.668605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 805/1608] 10.222.0.52 () {54 vars in 960 bytes} [Fri Oct 28 09:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:20.669832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.670205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.670573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.677348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4a3f5-9eac-49bd-84c2-28113578e706 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.701926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] MFA Rules not processed for user `88f1886cc5cc40ecb284b7a366ef1f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:20.703280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:20.741060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.742306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.742578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.742845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] PATH_INFO: `/v3/projects/500be47836b44a769f676e3e4bc00721/users/cdb5ce192c764fe4946542f5960ccd39/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.767508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=500be47836b44a769f676e3e4bc00721, user_id=cdb5ce192c764fe4946542f5960ccd39, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.770276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.790574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 806/1609] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:20.791061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 804/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:20 2022] PUT /identity/v3/projects/500be47836b44a769f676e3e4bc00721/users/cdb5ce192c764fe4946542f5960ccd39/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:20.791164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.791252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.801383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aa1e9b-dcd6-4d50-b32d-2fbb0f65817f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.849596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc2e8b-8add-4399-b870-f259d9414cee admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.854108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b4f7a-1c8d-4448-947e-8e6ed33e183a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.854714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b4f7a-1c8d-4448-947e-8e6ed33e183a service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.854983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b4f7a-1c8d-4448-947e-8e6ed33e183a service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.855672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b4f7a-1c8d-4448-947e-8e6ed33e183a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.861706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.862328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.862606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.862899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.863591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.865621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.868656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:20.889330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 807/1611] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:20.890147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:20.899929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1ff1-ffb5-44ac-a780-fd7bd9b38a07 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:20.904756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c533b0-9a3e-43fc-a5b6-56542e2d2255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:20.905398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c533b0-9a3e-43fc-a5b6-56542e2d2255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.905669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c533b0-9a3e-43fc-a5b6-56542e2d2255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.905934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c533b0-9a3e-43fc-a5b6-56542e2d2255 admin admin] PATH_INFO: `/v3/projects/ae5e7d531c0f4a33a869309af965e3e5/users/8bee2d5f599642788c32e0d8b8338c19/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.922474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c533b0-9a3e-43fc-a5b6-56542e2d2255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5e7d531c0f4a33a869309af965e3e5, user_id=8bee2d5f599642788c32e0d8b8338c19, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.925345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c533b0-9a3e-43fc-a5b6-56542e2d2255 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.931902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b4f7a-1c8d-4448-947e-8e6ed33e183a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:20.933499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b4f7a-1c8d-4448-947e-8e6ed33e183a service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:20.939915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 808/1612] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:20 2022] PUT /identity/v3/projects/ae5e7d531c0f4a33a869309af965e3e5/users/8bee2d5f599642788c32e0d8b8338c19/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:20.941153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51cb0b-4529-4e7c-a897-35cefe2f911f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.941435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51cb0b-4529-4e7c-a897-35cefe2f911f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.941705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51cb0b-4529-4e7c-a897-35cefe2f911f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.943969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 805/1613] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:20.945564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:20.945841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:20.946278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:20.993205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:20.994566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:20.995305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fd51cb0b-4529-4e7c-a897-35cefe2f911f None None] MFA Rules not processed for user `032b078fb1b1451ea594f6ecaf2fd366`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:20.997512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fd51cb0b-4529-4e7c-a897-35cefe2f911f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:21.017849 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:21.090313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 806/1614] 10.222.0.52 () {58 vars in 1057 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:21.090993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.101547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 809/1615] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:21.102647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c510d5-2f99-4c66-a507-208089787bfe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.102929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c510d5-2f99-4c66-a507-208089787bfe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.104074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c510d5-2f99-4c66-a507-208089787bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.127978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b6c510d5-2f99-4c66-a507-208089787bfe None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:21.132659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b6c510d5-2f99-4c66-a507-208089787bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:21.192233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7427637-58b7-41a1-8e39-db0dd0b32cc1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.199239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1a324-3037-41e1-b13e-e900de581fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.199885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1a324-3037-41e1-b13e-e900de581fed admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.200160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1a324-3037-41e1-b13e-e900de581fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.200427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1a324-3037-41e1-b13e-e900de581fed admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.201444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1a324-3037-41e1-b13e-e900de581fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.203331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b1a324-3037-41e1-b13e-e900de581fed admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.214266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 810/1616] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:21.216227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bb314-81ad-4ca2-afbd-14c7f18efe88 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.216607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bb314-81ad-4ca2-afbd-14c7f18efe88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.216951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39bb314-81ad-4ca2-afbd-14c7f18efe88 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.218917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 811/1617] 10.222.0.52 () {54 vars in 960 bytes} [Fri Oct 28 09:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:21.219787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bb314-81ad-4ca2-afbd-14c7f18efe88 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.236121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39bb314-81ad-4ca2-afbd-14c7f18efe88 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.241756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.242379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.242660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.242915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.243560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.245326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.249226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 807/1618] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:21.254959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.255648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.256259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.257267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 812/1619] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:21.258247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.272098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fdbc9-3771-44eb-8369-f091844daa38 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.282295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:21.284131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:21.296261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:21.354777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.355698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.355920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.356538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.391308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 808/1620] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:21.392370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.405374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f382b-a20c-47c0-9a75-d235289a63f8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.411060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb66f4-9c01-4108-b23d-1ffe0d251077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.412705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb66f4-9c01-4108-b23d-1ffe0d251077 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.412987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb66f4-9c01-4108-b23d-1ffe0d251077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.414083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb66f4-9c01-4108-b23d-1ffe0d251077 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.414803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb66f4-9c01-4108-b23d-1ffe0d251077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.416906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb66f4-9c01-4108-b23d-1ffe0d251077 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.418111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:21.440232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.442131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.458322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 809/1621] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:21.461160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.461820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.462393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.469966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 813/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:21.471057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.476104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:21.503179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:21.504729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:21.566422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603f8a6-6714-4867-b38e-4a4b329d80cb service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.595763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 810/1623] 10.222.0.52 () {58 vars in 1057 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:21.596817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.637179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.638120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.638806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.639243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.691872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60e947-70a7-4e0c-a7e1-8215040da37d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.700358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.701284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.701794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.702403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.703624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.706479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.714943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.716701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.728740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 814/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:21.729634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.735767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 811/1625] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:21.736824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.739629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5af88-f967-4ad1-a788-f6edee8d988f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.745002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.745777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.746061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.746332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] PATH_INFO: `/v3/projects/0fc2601672544130961bb2dd6bb0ad47/users/a2a9b22315814d4bb6bf4aa825ad56de/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.748191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088d9b9-f7f8-477a-8150-3e232b8b41f7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.752926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2c93b-5462-4c6a-a961-474d2f0fa3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.753695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2c93b-5462-4c6a-a961-474d2f0fa3fe service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.754291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2c93b-5462-4c6a-a961-474d2f0fa3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.754662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2c93b-5462-4c6a-a961-474d2f0fa3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.763779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc2601672544130961bb2dd6bb0ad47, user_id=a2a9b22315814d4bb6bf4aa825ad56de, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.765918 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2c93b-5462-4c6a-a961-474d2f0fa3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.766502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.767472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2c93b-5462-4c6a-a961-474d2f0fa3fe service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.774535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:21.783676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 815/1626] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:21 2022] PUT /identity/v3/projects/0fc2601672544130961bb2dd6bb0ad47/users/a2a9b22315814d4bb6bf4aa825ad56de/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:21.787709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.791816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 812/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:21.794485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553958b-5263-427e-b2d6-4fd9c034efc3 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.795261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553958b-5263-427e-b2d6-4fd9c034efc3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.795769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553958b-5263-427e-b2d6-4fd9c034efc3 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.797841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 813/1628] 10.222.0.52 () {54 vars in 912 bytes} [Fri Oct 28 09:45:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:21.800137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad936b-f5a4-4035-840f-6675877c9c5b None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.800915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad936b-f5a4-4035-840f-6675877c9c5b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.801619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad936b-f5a4-4035-840f-6675877c9c5b None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.804659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 814/1629] 10.222.0.52 () {54 vars in 912 bytes} [Fri Oct 28 09:45:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:21.808794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad936b-f5a4-4035-840f-6675877c9c5b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.820541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:21.883721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e30ee9-0e77-47ba-94f1-807faa0069a9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.888829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.889493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.889914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.890391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.891332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.894339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.914650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad936b-f5a4-4035-840f-6675877c9c5b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.924456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae58234-46d1-4cee-ade2-a72fe8726fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.925355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae58234-46d1-4cee-ade2-a72fe8726fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.925598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae58234-46d1-4cee-ade2-a72fe8726fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.926008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae58234-46d1-4cee-ade2-a72fe8726fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.927153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae58234-46d1-4cee-ade2-a72fe8726fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.929282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae58234-46d1-4cee-ade2-a72fe8726fe1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.938999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 816/1630] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:21.938999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:21.945256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 815/1631] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:21.952705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.953280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.953280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.959632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b092df-7b66-4a6d-818a-8475c2afc628 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:21.965283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910fbcf-8dfc-4626-81d6-2cc4ee1faf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:21.967109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910fbcf-8dfc-4626-81d6-2cc4ee1faf97 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.967783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910fbcf-8dfc-4626-81d6-2cc4ee1faf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.968285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910fbcf-8dfc-4626-81d6-2cc4ee1faf97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.969110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910fbcf-8dfc-4626-81d6-2cc4ee1faf97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:21.970854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910fbcf-8dfc-4626-81d6-2cc4ee1faf97 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:21.985225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 817/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:21.989736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:21.990444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:21.990938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:21.991682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:21.996110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:22.017755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:22.019404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:22.137178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 818/1633] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:22.138242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.143237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 816/1634] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:45:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:22.144902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.167086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e759ef4-5b55-4e7e-bc2c-0f6ef32788e9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.175876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.176832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.177651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.177651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.179673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:22.186419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaa9c2c-04c5-419c-a6ff-5e453b9b7f69 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.191659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:22.195124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.196051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.196546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.198477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] PATH_INFO: `/v3/projects/670258ffb4334ea6b3ca1c32bb09abfe/users/b2c0b3ae6d9b40338421e61069bc664a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.224635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 819/1635] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:22.227951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.232081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=670258ffb4334ea6b3ca1c32bb09abfe, user_id=b2c0b3ae6d9b40338421e61069bc664a, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:22.241223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:22.251427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9feff-b81a-470d-a059-03466ac9a8b9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.257234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.258159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.258397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.258737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] PATH_INFO: `/v3/projects/0fc2601672544130961bb2dd6bb0ad47/users/a2a9b22315814d4bb6bf4aa825ad56de/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.261259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc2601672544130961bb2dd6bb0ad47, user_id=a2a9b22315814d4bb6bf4aa825ad56de, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:22.264540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:22.292199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 820/1636] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:22 2022] PUT /identity/v3/projects/0fc2601672544130961bb2dd6bb0ad47/users/a2a9b22315814d4bb6bf4aa825ad56de/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:22.293544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.293753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 817/1637] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:22 2022] PUT /identity/v3/projects/670258ffb4334ea6b3ca1c32bb09abfe/users/b2c0b3ae6d9b40338421e61069bc664a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:22.295498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.385007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc602-a4a2-4e8f-a711-9fa2b0387ccf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.391097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79286-152c-4363-a131-b5e28af972f0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.395712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.396477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.396816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.397190 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.403056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f03d1-45b0-4bca-872f-02a1072e4a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.406501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f03d1-45b0-4bca-872f-02a1072e4a09 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.408433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f03d1-45b0-4bca-872f-02a1072e4a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.408765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f03d1-45b0-4bca-872f-02a1072e4a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.434097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:22.443711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:22.510191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:22.511796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:22.526114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 821/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:22.526965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.544636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f03d1-45b0-4bca-872f-02a1072e4a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:22.546686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f03d1-45b0-4bca-872f-02a1072e4a09 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:22.567484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 818/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:22.570194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.570720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.571506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.625529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] MFA Rules not processed for user `a2a9b22315814d4bb6bf4aa825ad56de`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:22.638795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:22.668441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf355e-9562-45fe-8a8f-e267db24cda4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.691790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.692961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:22.694426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:22.694782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] PATH_INFO: `/v3/projects/69a7cede1c214895a4e6e0eb47eb122b/users/198b2a477716424ebcd479a2e5606dfd/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:22.747056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a7cede1c214895a4e6e0eb47eb122b, user_id=198b2a477716424ebcd479a2e5606dfd, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:22.760511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:22.795274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 822/1640] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:22 2022] PUT /identity/v3/projects/69a7cede1c214895a4e6e0eb47eb122b/users/198b2a477716424ebcd479a2e5606dfd/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:22.795274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.802679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 819/1641] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:22.802679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:22.825748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f40489-d5fe-457b-9a28-1af03f77b652 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.984357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92639e3e-fab5-445a-a787-13ae6e8e3839 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:22.999002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:22.999832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.000373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.000569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.001337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:23.003166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:23.035769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:23.036726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 823/1642] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:23.036915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.037243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.037696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:23.037875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.039794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:23.051090 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:23.063507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1c2c8-3f80-408e-9771-896d61af7efc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:23.080861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780c903-122e-42fe-b59b-28cc6532bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:23.081561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c903-122e-42fe-b59b-28cc6532bbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.081926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c903-122e-42fe-b59b-28cc6532bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.082260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c903-122e-42fe-b59b-28cc6532bbe9 admin admin] PATH_INFO: `/v3/projects/670258ffb4334ea6b3ca1c32bb09abfe/users/b2c0b3ae6d9b40338421e61069bc664a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.087865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780c903-122e-42fe-b59b-28cc6532bbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670258ffb4334ea6b3ca1c32bb09abfe, user_id=b2c0b3ae6d9b40338421e61069bc664a, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:23.090583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780c903-122e-42fe-b59b-28cc6532bbe9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:23.110376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 820/1643] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:23.117037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:23.118863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 824/1644] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:23 2022] PUT /identity/v3/projects/670258ffb4334ea6b3ca1c32bb09abfe/users/b2c0b3ae6d9b40338421e61069bc664a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:23.133751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.133751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.133751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.162408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8dcace-2a86-4f30-89b4-bfa959bc983b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:23.190147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:23.196832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] MFA Rules not processed for user `b2c0b3ae6d9b40338421e61069bc664a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:23.202319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:23.294570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:23.390005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a3c20-1afc-4aad-a76b-33e2a2dc7814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:23.394924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a3c20-1afc-4aad-a76b-33e2a2dc7814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.398445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a3c20-1afc-4aad-a76b-33e2a2dc7814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.398970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a3c20-1afc-4aad-a76b-33e2a2dc7814 admin admin] PATH_INFO: `/v3/projects/69a7cede1c214895a4e6e0eb47eb122b/users/198b2a477716424ebcd479a2e5606dfd/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.407465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a3c20-1afc-4aad-a76b-33e2a2dc7814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a7cede1c214895a4e6e0eb47eb122b, user_id=198b2a477716424ebcd479a2e5606dfd, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:23.422902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a3c20-1afc-4aad-a76b-33e2a2dc7814 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:23.430945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 825/1645] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:23.431778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:23.472189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 821/1646] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:23 2022] PUT /identity/v3/projects/69a7cede1c214895a4e6e0eb47eb122b/users/198b2a477716424ebcd479a2e5606dfd/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:45:23.483355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.484342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.484342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.556554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] MFA Rules not processed for user `198b2a477716424ebcd479a2e5606dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:23.557849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:23.595864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a858a8f-b361-4424-95d4-490f4cc42e8b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:23.805562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:23.806353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.806634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.806958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:23.846537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 822/1647] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:23.854314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:23.872338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94217e43-d06b-45c6-ae54-c600b2450095 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:23.887505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:23.888191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:23.888467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:23.888740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:24.045673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:24.047839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:24.050324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:24.052024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:24.077482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 823/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:24.095065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 826/1649] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:24.264342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:24.277798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d49c84-6dee-43c5-82a9-287cc8abe9fb service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:24.287585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:24.290613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:24.293947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:24.294576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:24.322270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:24.326665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:24.359392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 824/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:24.720050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:24.727376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:24.735450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771e0a3-630d-4092-a5ef-46f328c2be4c service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:24.745450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:24.746092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:24.746446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:24.746641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:24.900660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:24.938005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f419c-83d0-4083-93a0-64830456f527 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:24.989477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa2f00-5361-4058-89a7-4496b1bdb127 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:24.989477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2f00-5361-4058-89a7-4496b1bdb127 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:24.989477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2f00-5361-4058-89a7-4496b1bdb127 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:24.989477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa2f00-5361-4058-89a7-4496b1bdb127 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:24.991720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:24.991720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:24.996599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 827/1651] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:25.021860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:25.073609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa2f00-5361-4058-89a7-4496b1bdb127 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:25.076393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa2f00-5361-4058-89a7-4496b1bdb127 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:25.088872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 825/1652] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:26.956627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:27.185466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7747-5874-41d5-90ae-79a6cc827a20 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:27.206309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:27.207470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:27.207785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:27.208081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] PATH_INFO: `/v3/users/f06db17158464566aae64758ef09edba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:27.213015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06db17158464566aae64758ef09edba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:27.218340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:27.348025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06db17158464566aae64758ef09edba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:27.374527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06db17158464566aae64758ef09edba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:27.418509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06db17158464566aae64758ef09edba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:27.419296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06db17158464566aae64758ef09edba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:27.438743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06db17158464566aae64758ef09edba'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:27.458129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 828/1653] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:26 2022] DELETE /identity/v3/users/f06db17158464566aae64758ef09edba => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:27.460354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:27.461277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:27.461712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:27.503785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:27.509499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:27.671950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 826/1654] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:27.756196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:27.757674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:27.836275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:27.966183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd66731-ca89-441c-b3f0-5d1c1e711df8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:27.988855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29345986-03ba-4e19-8ee6-95f1b04ae670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:27.989525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29345986-03ba-4e19-8ee6-95f1b04ae670 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:27.989809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29345986-03ba-4e19-8ee6-95f1b04ae670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:27.990079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29345986-03ba-4e19-8ee6-95f1b04ae670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:28.027406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29345986-03ba-4e19-8ee6-95f1b04ae670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:28.029009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29345986-03ba-4e19-8ee6-95f1b04ae670 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:28.068913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 829/1655] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:45:29.504463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:29.508519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:29.570200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe3724-9992-413b-81a6-09350027da2b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:29.596083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:29.597857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:29.605427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:29.608873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] PATH_INFO: `/v3/projects/7bf643c5c7aa47468b2796ef67ad5321` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:29.612911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf643c5c7aa47468b2796ef67ad5321)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:29.624371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:29.817256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf643c5c7aa47468b2796ef67ad5321', 'request_id': 'req-a06b2923-a754-4f05-84ab-14aef775e673'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:29.830948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf643c5c7aa47468b2796ef67ad5321', 'request_id': 'req-a06b2923-a754-4f05-84ab-14aef775e673'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:29.843319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf643c5c7aa47468b2796ef67ad5321', 'request_id': 'req-a06b2923-a754-4f05-84ab-14aef775e673'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:29.880776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] The token cache is being invalidate because project 7bf643c5c7aa47468b2796ef67ad5321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:29.881366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:29.882489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:29.887189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 827/1656] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:29 2022] DELETE /identity/v3/projects/7bf643c5c7aa47468b2796ef67ad5321 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:29.892348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e304e-a2d4-4855-9d1e-a77dfa686f68 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:29.893758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e304e-a2d4-4855-9d1e-a77dfa686f68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:29.894603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e304e-a2d4-4855-9d1e-a77dfa686f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:29.941292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:29.943617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:29.945795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2f7e304e-a2d4-4855-9d1e-a77dfa686f68 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:29.961843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f7e304e-a2d4-4855-9d1e-a77dfa686f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:30.098234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b2923-a754-4f05-84ab-14aef775e673 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:30.112506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:30.115510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.116142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.116561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.119170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:30.130585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:30.166154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 830/1657] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:30.174784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.175308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.175729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.222921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:30.226974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:30.251668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 831/1658] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:30.252699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:30.278260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d001a7-d597-4d0b-a9fb-02e6f9052cc2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:30.284962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:30.289747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:30.290978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:30.291648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.291965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.292254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.294269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:30.294620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:30.302766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:30.333777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 832/1659] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:30.334922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:30.335268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 828/1660] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:30.347446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f10fbb-f227-4df2-a3d0-f349556af943 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:30.353409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:30.355406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.356040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.356388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.357079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:30.359556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:30.434051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 833/1661] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:30 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:30.435095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:30.480048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e477a56-baa4-44ca-a3bb-c87ddc477d1c service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:30.749364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:30.749928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.750200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.750463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.751120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:30.760054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:30.818756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 829/1662] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:30 2022] POST /identity/v3/users => generated 303 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:30.823363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:30.853439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb04247-429d-4109-81fb-3cb1ec024923 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:30.879612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:30.880430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.884083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.884545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.885875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:30.887887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:30.925766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 834/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:30.927964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:30.957234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268be83-fe61-490c-aa86-05b2129f21f9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:30.992641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bcdfe-f7af-4dc7-b616-2e1bfc433593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:30.995386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bcdfe-f7af-4dc7-b616-2e1bfc433593 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:30.996014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bcdfe-f7af-4dc7-b616-2e1bfc433593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:30.996656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bcdfe-f7af-4dc7-b616-2e1bfc433593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:30.997594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bcdfe-f7af-4dc7-b616-2e1bfc433593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:30.999917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bcdfe-f7af-4dc7-b616-2e1bfc433593 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:31.031217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:31.035152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 830/1664] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:31.062287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820a916-704b-4d59-b5b7-cf9723f3729a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:31.100038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:31.104532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:31.108001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:31.108600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] PATH_INFO: `/v3/projects/195c2ade17064592acc581ed6d3fedba/users/394f6f90975d4fe3bf5123182979288c/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:31.198633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195c2ade17064592acc581ed6d3fedba, user_id=394f6f90975d4fe3bf5123182979288c, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:31.205534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:31.282830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 835/1665] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:31 2022] PUT /identity/v3/projects/195c2ade17064592acc581ed6d3fedba/users/394f6f90975d4fe3bf5123182979288c/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:31.288894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f9987-9cbc-416d-8897-d9c393e0a73a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:31.289352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f9987-9cbc-416d-8897-d9c393e0a73a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:31.289848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f9987-9cbc-416d-8897-d9c393e0a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:31.358747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bc8f9987-9cbc-416d-8897-d9c393e0a73a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:31.366247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc8f9987-9cbc-416d-8897-d9c393e0a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:31.622433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 831/1666] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:31.662445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:31.669202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:31.966425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:31.994388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ef086-a5f2-47f7-900f-724af6de12d2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:32.012781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c75e34-42e2-4efd-a5b2-9c0b65c6f252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:32.014106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c75e34-42e2-4efd-a5b2-9c0b65c6f252 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:32.014770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c75e34-42e2-4efd-a5b2-9c0b65c6f252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:32.015415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c75e34-42e2-4efd-a5b2-9c0b65c6f252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:32.048116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:32.049390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:32.050065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:32.101101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c75e34-42e2-4efd-a5b2-9c0b65c6f252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:32.104207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c75e34-42e2-4efd-a5b2-9c0b65c6f252 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:32.170108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 836/1667] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:32.197117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] MFA Rules not processed for user `5e01f518dbd947bfafa6aecdccd9cfd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:32.199164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:32.236619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:32.480087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 832/1668] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:32.504614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3e8fb-1139-436d-b5e6-343c5c6cb870 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:32.505059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3e8fb-1139-436d-b5e6-343c5c6cb870 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:32.505556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3e8fb-1139-436d-b5e6-343c5c6cb870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:32.603547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-31d3e8fb-1139-436d-b5e6-343c5c6cb870 None None] MFA Rules not processed for user `5e01f518dbd947bfafa6aecdccd9cfd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:32.605552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-31d3e8fb-1139-436d-b5e6-343c5c6cb870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:32.666977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 837/1669] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:32.689018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:32.690819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:32.929722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736fa38-7cff-48c4-9442-77f825cedce8 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:32.950823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35401b-1c4b-48c5-85cc-e4fb19878c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:32.952114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35401b-1c4b-48c5-85cc-e4fb19878c14 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:32.952882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35401b-1c4b-48c5-85cc-e4fb19878c14 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:32.953622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35401b-1c4b-48c5-85cc-e4fb19878c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:32.966071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35401b-1c4b-48c5-85cc-e4fb19878c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:32.968310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35401b-1c4b-48c5-85cc-e4fb19878c14 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:33.007671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 833/1670] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:34.374238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568eb7a4-42d7-4eb3-a179-0bcb2fbe170b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:34.375834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568eb7a4-42d7-4eb3-a179-0bcb2fbe170b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:34.376740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568eb7a4-42d7-4eb3-a179-0bcb2fbe170b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:34.379806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 838/1671] 10.222.0.52 () {54 vars in 912 bytes} [Fri Oct 28 09:45:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:34.387598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:34.388311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:34.388311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:34.612787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] MFA Rules not processed for user `b07b6ea0a9a7475fbcc8d6baf537cc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:34.619264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:34.652144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:34.657427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:34.658180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:34.674985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 834/1672] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:45:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:34.697053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:34.737516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a1890-f7cc-4216-a45f-0c595d4a4a1b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:34.745898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:34.746973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:34.747089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] MFA Rules not processed for user `394f6f90975d4fe3bf5123182979288c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:34.747642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:34.748158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:34.751006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:34.771939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:34.773869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:34.788241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 835/1673] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:35.000028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 839/1674] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:35.004857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:35.035297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1659f-cd78-44da-8e3b-ea4989a87650 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:35.056790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:35.057607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:35.058014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:35.058392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:35.060467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:35.065208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:35.174834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 836/1675] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:35 2022] POST /identity/v3/projects => generated 403 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:35.180455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:35.213559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de5178-f264-4cfe-a997-039c3814131c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:35.381292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:35.387841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:35.389625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:35.390046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:35.394492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:35.403197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:35.496441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:35.497947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 840/1676] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:35 2022] POST /identity/v3/users => generated 308 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:35.511650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde606c-c4b0-402d-904a-3989ee06793f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:35.523347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:35.524025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:35.524323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:35.524562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:35.525230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:35.526889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:35.557098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 837/1677] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:45:35.563025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:35.601981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f05bd6-f958-44e9-b185-a908637cde57 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:35.620061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:35.621754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:35.623625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:35.625581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] PATH_INFO: `/v3/projects/85f64e75af344a9e9657becff128eedc/users/bc1e2b36dc1c4424bd0b78f6468be077/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:35.698651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f64e75af344a9e9657becff128eedc, user_id=bc1e2b36dc1c4424bd0b78f6468be077, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:35.709853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:35.754332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 841/1678] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:35 2022] PUT /identity/v3/projects/85f64e75af344a9e9657becff128eedc/users/bc1e2b36dc1c4424bd0b78f6468be077/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:45:35.758730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:35.774492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42300d6-3262-4d4e-9169-926b41344a05 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:35.892823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:35.943680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:35.944793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:35.945128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:35.945453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:35.946287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:35.948358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:35.978859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 838/1679] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:45:35.984623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:36.017895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401c292-d606-48e3-b1ee-ee18fa227308 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:36.033796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:36.045099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:36.046822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:36.047628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] PATH_INFO: `/v3/projects/85f64e75af344a9e9657becff128eedc/users/bc1e2b36dc1c4424bd0b78f6468be077/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:36.052681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] RBAC: Authorizing `identity:create_grant(project_id=85f64e75af344a9e9657becff128eedc, user_id=bc1e2b36dc1c4424bd0b78f6468be077, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:36.062963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:36.096242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 842/1680] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:35 2022] PUT /identity/v3/projects/85f64e75af344a9e9657becff128eedc/users/bc1e2b36dc1c4424bd0b78f6468be077/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:36.176102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:36.179266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:36.453905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87b351-8d25-40c7-8164-dee84cae2d7e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:36.647466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:36.648505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:36.648505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:36.649931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:36.657933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:36.697555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:36.710448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:36.744661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 839/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:37.579861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:37.584206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:37.852098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:38.037732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b031f56-b2bc-4ee2-91ad-a259e92a269d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:38.056972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:38.058010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:38.058484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:38.058948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] PATH_INFO: `/v3/users/52a7bc3f048c47f6b3d8616a2b9bd3f2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:38.066082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a7bc3f048c47f6b3d8616a2b9bd3f2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:38.070516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:38.208169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:38.210752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:38.299595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a7bc3f048c47f6b3d8616a2b9bd3f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.305977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23e8e4-611f-4019-8815-0c668ee1ced2 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:38.333236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:38.335814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:38.336658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:38.337435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] PATH_INFO: `/v3/users/294d317eec794729a7ffd85d42288b21` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:38.341359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=294d317eec794729a7ffd85d42288b21)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:38.345441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: WARNING py.warnings [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:45:38.345441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: warnings.warn(msg) Oct 28 09:45:38.345441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: Oct 28 09:45:38.349115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a7bc3f048c47f6b3d8616a2b9bd3f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.352000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:38.405561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a7bc3f048c47f6b3d8616a2b9bd3f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.406830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a7bc3f048c47f6b3d8616a2b9bd3f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.443564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b892f0a5-3e31-41cd-bf29-5c334b08da58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a7bc3f048c47f6b3d8616a2b9bd3f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.471886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 843/1682] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:37 2022] DELETE /identity/v3/users/52a7bc3f048c47f6b3d8616a2b9bd3f2 => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:45:38.480940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:38.481497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:38.482173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:38.565416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:38.567591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:38.579934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294d317eec794729a7ffd85d42288b21'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.618660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294d317eec794729a7ffd85d42288b21'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.622734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:38.661463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294d317eec794729a7ffd85d42288b21'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.663568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294d317eec794729a7ffd85d42288b21'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.728625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d071515b-59ba-462e-bed6-48c8bf5dc126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294d317eec794729a7ffd85d42288b21'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:38.753444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 840/1683] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:38 2022] DELETE /identity/v3/users/294d317eec794729a7ffd85d42288b21 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:38.759846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:38.760140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:38.760421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:38.797300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:38.798168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:38.867357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 844/1684] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:38.879725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:38.882747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:39.042590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 841/1685] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:39.050087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:39.051607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:39.178109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:39.207046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b073e-5c3d-432f-9af5-fbf86254d0b9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:39.234476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f9df9-77c7-4c65-8e01-df88537ea856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:39.236081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f9df9-77c7-4c65-8e01-df88537ea856 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:39.236860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f9df9-77c7-4c65-8e01-df88537ea856 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:39.237379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f9df9-77c7-4c65-8e01-df88537ea856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:39.253811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191aa4b-d16a-4508-b1d6-0b465445cfe7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:39.263231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f9df9-77c7-4c65-8e01-df88537ea856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:39.269984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:39.270292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f9df9-77c7-4c65-8e01-df88537ea856 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:39.271042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:39.271502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:39.271997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:39.300420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:39.307009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:39.309053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 845/1686] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:45:39.348986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 842/1687] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:45:39.608821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:39.609438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:39.610142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:39.800265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] MFA Rules not processed for user `8bee2d5f599642788c32e0d8b8338c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:39.801584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:40.012503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 846/1688] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:40.035500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:40.038135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:40.368864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed08b6-e987-4b7b-ab6d-2b8c4c581b7f service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:40.383461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de2e5ab-fd0e-4ca2-9d4d-8066b1790672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:40.387269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2e5ab-fd0e-4ca2-9d4d-8066b1790672 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:40.388049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2e5ab-fd0e-4ca2-9d4d-8066b1790672 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:40.389828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de2e5ab-fd0e-4ca2-9d4d-8066b1790672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:40.438063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2e5ab-fd0e-4ca2-9d4d-8066b1790672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:40.446466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de2e5ab-fd0e-4ca2-9d4d-8066b1790672 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:40.481680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:40.509084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 843/1689] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:40.606968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:40.609973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:40.646670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf58eb-23a4-4511-a7ca-f477b28506ea None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:40.675942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:40.677978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:40.678716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:40.681919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] PATH_INFO: `/v3/projects/28039617399a4a5094039022d287ccd6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:40.688803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28039617399a4a5094039022d287ccd6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:40.694914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: WARNING py.warnings [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 28 09:45:40.694914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: warnings.warn(msg) Oct 28 09:45:40.694914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: Oct 28 09:45:40.700271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:40.930504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28039617399a4a5094039022d287ccd6', 'request_id': 'req-37b9472d-d77c-431c-8aa4-e63c33179a71'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:40.948768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28039617399a4a5094039022d287ccd6', 'request_id': 'req-37b9472d-d77c-431c-8aa4-e63c33179a71'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:40.965401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28039617399a4a5094039022d287ccd6', 'request_id': 'req-37b9472d-d77c-431c-8aa4-e63c33179a71'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:41.051012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] The token cache is being invalidate because project 28039617399a4a5094039022d287ccd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:41.051973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:41.052496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37b9472d-d77c-431c-8aa4-e63c33179a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:41.055322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 847/1690] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:40 2022] DELETE /identity/v3/projects/28039617399a4a5094039022d287ccd6 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:41.060897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f837ee-4a2e-4220-b37a-38c1d5785bad None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.061385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f837ee-4a2e-4220-b37a-38c1d5785bad None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.061855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f837ee-4a2e-4220-b37a-38c1d5785bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.109546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-19f837ee-4a2e-4220-b37a-38c1d5785bad None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:41.111031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-19f837ee-4a2e-4220-b37a-38c1d5785bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:41.269006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.269510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.269936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.281633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 844/1691] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:41.295958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.296446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.296934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.319323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:41.324470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:41.349947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 848/1692] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:41.354093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:41.376081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b8787-122c-4476-9817-9586938c0fb5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:41.391602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:41.393713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.394098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.395176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.396424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:41.398961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:41.401501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:41.426290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:41.434897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:41.435934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 849/1693] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:41.437093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:41.458368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c293445-08c9-445e-ad8c-628236b7fb71 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:41.474734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2927c9-78ee-49c9-a956-dcaeee9f01a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:41.476196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2927c9-78ee-49c9-a956-dcaeee9f01a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.476607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2927c9-78ee-49c9-a956-dcaeee9f01a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.477282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2927c9-78ee-49c9-a956-dcaeee9f01a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.478072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2927c9-78ee-49c9-a956-dcaeee9f01a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:41.482388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2927c9-78ee-49c9-a956-dcaeee9f01a2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:41.550193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 850/1694] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:41 2022] POST /identity/v3/projects => generated 405 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:41.555336 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.555786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.556501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.611603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] MFA Rules not processed for user `bc1e2b36dc1c4424bd0b78f6468be077`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:41.615155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:41.635652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 845/1695] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:45:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:41.637736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:41.649749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:41.674551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68eba4-bac2-4178-ba02-5849b35442a7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:41.862154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 851/1696] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:41 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:41.891615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:41.894998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:41.950279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:41.953271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:41.953808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:41.956869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:41.959256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:41.965823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:42.018390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 846/1697] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:41 2022] POST /identity/v3/users => generated 303 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:42.025346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:42.036261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:42.052309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ef7ec-8314-4c48-a90f-b6a30187d82f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:42.070389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:42.071313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:42.071633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:42.072099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:42.072866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:42.076240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:42.115770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 847/1698] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:42.120573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:42.172061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3ec00-e7f6-4477-b7c3-2363855f5d15 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:42.206891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:42.207698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:42.208160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:42.208441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:42.209255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:42.214828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:42.261594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89c036-f08f-4671-aa6e-04ba353e914f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:42.263788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 848/1699] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:42 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:42.267277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:42.275361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:42.276476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:42.276934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:42.277281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:42.298619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c401f7-dcae-46a7-870e-a8d097f48127 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:42.313138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b4b2a-3f98-4f0e-b761-28a20d229a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:42.313736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4b2a-3f98-4f0e-b761-28a20d229a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:42.313995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4b2a-3f98-4f0e-b761-28a20d229a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:42.314447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b4b2a-3f98-4f0e-b761-28a20d229a57 admin admin] PATH_INFO: `/v3/projects/0c449f6dfb37478c855a0d4a50943451/users/1b347224d7f446568300515cff5b6546/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:42.364341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b4b2a-3f98-4f0e-b761-28a20d229a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c449f6dfb37478c855a0d4a50943451, user_id=1b347224d7f446568300515cff5b6546, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:42.367200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b4b2a-3f98-4f0e-b761-28a20d229a57 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:42.402046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 849/1700] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:42 2022] PUT /identity/v3/projects/0c449f6dfb37478c855a0d4a50943451/users/1b347224d7f446568300515cff5b6546/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:42.426393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:42.427372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:42.427372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:42.559107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] MFA Rules not processed for user `1b347224d7f446568300515cff5b6546`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:42.570304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:42.661637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:42.677423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:42.747387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 852/1701] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:45:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 857 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:45:42.748993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:42.750711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:42.792009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794d96c-a74d-4151-88df-9afb4b021df8 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:42.936347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:43.051872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 850/1702] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 627 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:43.059109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:43.146999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:43.148967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:43.152048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:43.153708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] PATH_INFO: `/v3/projects/10eecff17fe242a7b93f7d5e88aee49d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:43.156501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10eecff17fe242a7b93f7d5e88aee49d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:43.171491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:43.303051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57d7ed-b743-4beb-8a87-60988d08cf05 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:43.313944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:43.315897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:43.316392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:43.316661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:43.319664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:43.325370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:43.406972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 851/1703] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:43 2022] POST /identity/v3/projects => generated 403 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:43.408138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:43.533951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:43.564232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10eecff17fe242a7b93f7d5e88aee49d', 'request_id': 'req-f52134c9-86ad-465b-a0a8-f3763bd889c4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:43.596757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10eecff17fe242a7b93f7d5e88aee49d', 'request_id': 'req-f52134c9-86ad-465b-a0a8-f3763bd889c4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:43.635238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10eecff17fe242a7b93f7d5e88aee49d', 'request_id': 'req-f52134c9-86ad-465b-a0a8-f3763bd889c4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:43.666731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76881f-6912-4cef-8096-cffd9fb8018e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:43.707373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] The token cache is being invalidate because project 10eecff17fe242a7b93f7d5e88aee49d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:43.716515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:43.724543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:43.727216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 853/1704] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:42 2022] DELETE /identity/v3/projects/10eecff17fe242a7b93f7d5e88aee49d => generated 0 bytes in 979 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:43.764713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:43.784916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:44.033438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:44.034510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:44.034955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:44.035573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:44.043468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:44.054346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:44.145787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 852/1705] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:43 2022] POST /identity/v3/users => generated 308 bytes in 738 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:44.151702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:44.154808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:44.162538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52134c9-86ad-465b-a0a8-f3763bd889c4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:44.190500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:44.193107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:44.196099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:44.198544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] PATH_INFO: `/v3/users/a2a9b22315814d4bb6bf4aa825ad56de` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:44.206679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a9b22315814d4bb6bf4aa825ad56de)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:44.216582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:44.245134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d647f2-70d6-4019-8119-4725def1af6e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:44.276037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:44.277104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:44.277591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:44.289303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:44.293086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:44.307689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:44.393299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 853/1706] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:44.400124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:44.429754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a9b22315814d4bb6bf4aa825ad56de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:44.443386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a9b22315814d4bb6bf4aa825ad56de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:44.454277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7436b49-86b1-4975-89a6-e1834ca393f6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:44.471120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:44.489151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:44.490348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:44.490836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:44.494236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] PATH_INFO: `/v3/projects/cba686e736b64257a940b10b0b7e3121/users/88fc5d92b21b4b889f090611ee144f29/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:44.527804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a9b22315814d4bb6bf4aa825ad56de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:44.532787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a9b22315814d4bb6bf4aa825ad56de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:44.589904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a9b22315814d4bb6bf4aa825ad56de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:44.598635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba686e736b64257a940b10b0b7e3121, user_id=88fc5d92b21b4b889f090611ee144f29, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:44.621880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:44.626309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 854/1707] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:43 2022] DELETE /identity/v3/users/a2a9b22315814d4bb6bf4aa825ad56de => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:44.679875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 854/1708] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:44 2022] PUT /identity/v3/projects/cba686e736b64257a940b10b0b7e3121/users/88fc5d92b21b4b889f090611ee144f29/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:44.683910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:44.686612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:44.933142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ff817-f80e-4311-9037-048646bb7059 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:45.095256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4572acb-f95e-4f26-9b70-eb52e4b40025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:45.096981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4572acb-f95e-4f26-9b70-eb52e4b40025 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:45.097302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4572acb-f95e-4f26-9b70-eb52e4b40025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:45.097543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4572acb-f95e-4f26-9b70-eb52e4b40025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:45.098245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4572acb-f95e-4f26-9b70-eb52e4b40025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:45.100192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4572acb-f95e-4f26-9b70-eb52e4b40025 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:45.138981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 855/1709] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:45.143727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:45.211473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec2a74b-900f-46bc-9f42-05c4ea7d0c4d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:45.242540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:45.243682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:45.244216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:45.244698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] PATH_INFO: `/v3/projects/cba686e736b64257a940b10b0b7e3121/users/88fc5d92b21b4b889f090611ee144f29/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:45.264390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba686e736b64257a940b10b0b7e3121, user_id=88fc5d92b21b4b889f090611ee144f29, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:45.288979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:45.372144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 855/1710] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:45 2022] PUT /identity/v3/projects/cba686e736b64257a940b10b0b7e3121/users/88fc5d92b21b4b889f090611ee144f29/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:45.387444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:45.387893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:45.388247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:45.485835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] MFA Rules not processed for user `88fc5d92b21b4b889f090611ee144f29`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:45.498253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:45.727707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 856/1711] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:45 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:45.743479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:45.745326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:45.944388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407cb54-0875-4ced-8261-4f6f54e62f3e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:45.948599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8d51a-90cf-47ec-9f78-293babb55e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:45.949248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8d51a-90cf-47ec-9f78-293babb55e38 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:45.949570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8d51a-90cf-47ec-9f78-293babb55e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:45.950680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8d51a-90cf-47ec-9f78-293babb55e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:45.963040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8d51a-90cf-47ec-9f78-293babb55e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:45.967722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8d51a-90cf-47ec-9f78-293babb55e38 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:45.997204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 856/1712] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:45 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:46.384403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:46.400729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:46.401280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:46.402293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:46.404398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:46.585452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] MFA Rules not processed for user `b07b6ea0a9a7475fbcc8d6baf537cc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:46.604732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:46.752222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:46.838344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981de8d-1b2d-4c26-b6a4-4d0db32e8830 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:46.890728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:46.893919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:46.895026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:46.895839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] PATH_INFO: `/v3/users/1b347224d7f446568300515cff5b6546` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:46.909611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b347224d7f446568300515cff5b6546)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:46.917151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:46.988225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 857/1713] 10.222.0.58 () {58 vars in 1015 bytes} [Fri Oct 28 09:45:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 590 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:47.116062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b347224d7f446568300515cff5b6546'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:47.142190 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b347224d7f446568300515cff5b6546'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:47.150058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:47.162323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:47.206124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae3802-b878-490e-82e4-524fd54d1775 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:47.222831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d594a8-88fa-4336-9004-6929a92e3637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:47.223973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d594a8-88fa-4336-9004-6929a92e3637 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:47.224502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d594a8-88fa-4336-9004-6929a92e3637 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:47.225241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d594a8-88fa-4336-9004-6929a92e3637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:47.247250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b347224d7f446568300515cff5b6546'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:47.267824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b347224d7f446568300515cff5b6546'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:47.282919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d594a8-88fa-4336-9004-6929a92e3637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:47.289384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d594a8-88fa-4336-9004-6929a92e3637 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:47.301178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-74b08f65-1bf9-459c-8e83-14d8decfcf31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b347224d7f446568300515cff5b6546'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:47.321602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 858/1714] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:47.328246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f0289-1105-41d2-afe6-0845c4983af6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:47.332316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f0289-1105-41d2-afe6-0845c4983af6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:47.333159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f0289-1105-41d2-afe6-0845c4983af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:47.349460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 857/1715] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:46 2022] DELETE /identity/v3/users/1b347224d7f446568300515cff5b6546 => generated 0 bytes in 962 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:47.352972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:47.354980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:47.356026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:47.437646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:47.443047 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:47.454888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-279f0289-1105-41d2-afe6-0845c4983af6 None None] MFA Rules not processed for user `cdb5ce192c764fe4946542f5960ccd39`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:47.459209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-279f0289-1105-41d2-afe6-0845c4983af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:47.732018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 858/1716] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:47.746873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 859/1717] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:47.768004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:47.769733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:47.799962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379efb8-dde6-43d4-80d4-40717db7155e None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:47.801695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379efb8-dde6-43d4-80d4-40717db7155e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:47.802466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379efb8-dde6-43d4-80d4-40717db7155e None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:47.808070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 860/1718] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:45:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:47.855222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:47.855222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:47.855685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:47.856325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:47.968593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:47.971013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:47.979910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce43715-93d6-46c4-b96d-349e350042b1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:47.994379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:47.994691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:47.995010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:47.995237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:48.014006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:48.019151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:48.052818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 859/1719] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:48.056206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:48.062205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:48.265067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c39fe9a-3d3d-4f54-9a27-721466ca63f3 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:48.277589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:48.278510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:48.278990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:48.279450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:48.300969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:48.303184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:48.350184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 860/1720] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:48.442590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 861/1721] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:45:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 589 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:48.549421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:48.551610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:48.591543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532a8d4-01c2-4f6f-8084-3e2cff2b28a1 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:48.609880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:48.611286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:48.611856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:48.612457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:48.649904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:48.653236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:48.704385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 861/1722] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:50.654897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:50.657571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:50.699177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc067480-ac92-4fff-874e-7f5be593d623 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:50.723665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:50.726203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:50.728415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:50.729337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] PATH_INFO: `/v3/projects/0fc2601672544130961bb2dd6bb0ad47` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:50.734064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc2601672544130961bb2dd6bb0ad47)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:50.744957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:50.824943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:50.827436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:50.879575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed9c30-60e7-44d5-b5fd-2ccc829639db service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:50.912380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:50.913079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:50.913675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:50.914062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:50.914295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] PATH_INFO: `/v3/users/88f1886cc5cc40ecb284b7a366ef1f7e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:50.920409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88f1886cc5cc40ecb284b7a366ef1f7e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:50.926294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:50.941960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc2601672544130961bb2dd6bb0ad47', 'request_id': 'req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:50.959194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc2601672544130961bb2dd6bb0ad47', 'request_id': 'req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:50.986467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc2601672544130961bb2dd6bb0ad47', 'request_id': 'req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.038005 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] The token cache is being invalidate because project 0fc2601672544130961bb2dd6bb0ad47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:51.039664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.042425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3f905e6a-8d40-47b8-b20e-cc877d11ec7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.046166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 862/1723] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:50 2022] DELETE /identity/v3/projects/0fc2601672544130961bb2dd6bb0ad47 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:51.053155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf33eef-6e19-43eb-b09b-c20fdb99c5e3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:51.057994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf33eef-6e19-43eb-b09b-c20fdb99c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:51.059476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf33eef-6e19-43eb-b09b-c20fdb99c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:51.105366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88f1886cc5cc40ecb284b7a366ef1f7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.126217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88f1886cc5cc40ecb284b7a366ef1f7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.128968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5cf33eef-6e19-43eb-b09b-c20fdb99c5e3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:51.132325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5cf33eef-6e19-43eb-b09b-c20fdb99c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:51.167133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88f1886cc5cc40ecb284b7a366ef1f7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.169368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88f1886cc5cc40ecb284b7a366ef1f7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.195595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12af26c7-1aca-469b-85cc-dfa2c5219487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88f1886cc5cc40ecb284b7a366ef1f7e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:51.213840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 862/1724] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:50 2022] DELETE /identity/v3/users/88f1886cc5cc40ecb284b7a366ef1f7e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:51.216633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:51.217211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:51.217670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:51.263177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:51.265622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:51.282298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:51.410888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 863/1725] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:51.420063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:51.420486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:51.420795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:51.489459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:51.493878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:51.592081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 863/1726] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:51.660499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 864/1727] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:51.667516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:51.681980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:51.683824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:51.695843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba75943-d277-466f-bc1e-7a984aebbaba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:51.726409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:51.727538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:51.727913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:51.728160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:51.728780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:51.731215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:51.740044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:51.796639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 864/1728] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:51.800075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:51.831122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad7a8f-43c9-492d-b813-3a501fab131d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:51.844534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:51.845810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:51.846098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:51.846548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:51.847239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:51.857997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:51.888312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1559e-9ad5-4465-9d6f-34d19a20c33f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:51.901248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:51.904168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:51.905057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:51.906125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:51.937273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:51.943515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 865/1729] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:51 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:51.944003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:51.944847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:51.977643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 865/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:52.018246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:52.021642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:52.066258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019d033-b31d-4620-8f6e-e667e9bf0b07 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:52.111274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:52.228634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18fe1c8-edc5-47f7-8291-c9e95cf30a08 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:52.238676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:52.239451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:52.239722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:52.240276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] PATH_INFO: `/v3/projects/0c449f6dfb37478c855a0d4a50943451` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:52.241549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c449f6dfb37478c855a0d4a50943451)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:52.243571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:52.244512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:52.245396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:52.245912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:52.246372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:52.247231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:52.249865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:52.350224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 866/1731] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:51 2022] POST /identity/v3/users => generated 283 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:45:52.351854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:52.384869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4f113-976a-4037-b5ae-752b31a57cb3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:52.480132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c449f6dfb37478c855a0d4a50943451', 'request_id': 'req-9de94657-c04e-444f-8825-d62129564afa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:52.531344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c449f6dfb37478c855a0d4a50943451', 'request_id': 'req-9de94657-c04e-444f-8825-d62129564afa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:52.544931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c449f6dfb37478c855a0d4a50943451', 'request_id': 'req-9de94657-c04e-444f-8825-d62129564afa'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:52.574540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] The token cache is being invalidate because project 0c449f6dfb37478c855a0d4a50943451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:52.574845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:52.578771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:52.587036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 866/1732] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:52 2022] DELETE /identity/v3/projects/0c449f6dfb37478c855a0d4a50943451 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:52.588223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:52.589745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:52.663449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:52.668570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:52.669011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:52.672959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:52.673938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:52.677239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:52.743587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 867/1733] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:52.745364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:52.748555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:52.780004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:52.798013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ab018-efab-41f6-be83-6f8e21ced512 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:52.822023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:52.827804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:52.829393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:52.832354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:52.833925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:52.838963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:52.855903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de94657-c04e-444f-8825-d62129564afa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:52.873477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:52.876121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:52.878341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:52.878992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] PATH_INFO: `/v3/users/88fc5d92b21b4b889f090611ee144f29` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:52.883290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fc5d92b21b4b889f090611ee144f29)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:52.890037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:52.897427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 868/1734] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:52 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:45:52.905210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:52.954209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc4734-e1e9-46bf-aeb5-3296002868e2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:52.971338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ec781-d65e-4d7c-9d4e-ec829b571359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:52.972481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec781-d65e-4d7c-9d4e-ec829b571359 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:52.973006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec781-d65e-4d7c-9d4e-ec829b571359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:52.973537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ec781-d65e-4d7c-9d4e-ec829b571359 admin admin] PATH_INFO: `/v3/projects/8fa541e8887844b3a5bccbfa866f63a8/users/1e5753560e9145bf8e922a422e9cc0da/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:53.011269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:53.013271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ec781-d65e-4d7c-9d4e-ec829b571359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa541e8887844b3a5bccbfa866f63a8, user_id=1e5753560e9145bf8e922a422e9cc0da, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:53.017739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ec781-d65e-4d7c-9d4e-ec829b571359 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:53.019955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fc5d92b21b4b889f090611ee144f29'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:53.043316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fc5d92b21b4b889f090611ee144f29'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:53.055566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fc5d92b21b4b889f090611ee144f29'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:53.056565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fc5d92b21b4b889f090611ee144f29'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:53.079792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 869/1735] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:52 2022] PUT /identity/v3/projects/8fa541e8887844b3a5bccbfa866f63a8/users/1e5753560e9145bf8e922a422e9cc0da/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:45:53.080331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bd414771-4f4a-4884-89c0-d9797460416d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fc5d92b21b4b889f090611ee144f29'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:53.090416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:53.091097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:53.091733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:53.107490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 867/1736] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:52 2022] DELETE /identity/v3/users/88fc5d92b21b4b889f090611ee144f29 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:45:53.227540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] MFA Rules not processed for user `1e5753560e9145bf8e922a422e9cc0da`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:53.231112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:53.283732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb646c62-e007-4ac6-8b13-cd5adb3bcbaf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:53.284220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb646c62-e007-4ac6-8b13-cd5adb3bcbaf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:53.284659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb646c62-e007-4ac6-8b13-cd5adb3bcbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:53.343662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-eb646c62-e007-4ac6-8b13-cd5adb3bcbaf None None] MFA Rules not processed for user `8bee2d5f599642788c32e0d8b8338c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:53.349141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-eb646c62-e007-4ac6-8b13-cd5adb3bcbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:53.445236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 870/1737] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:53.446706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:53.450015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:53.537273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 868/1738] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:53.725531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb982285-f765-4233-b98a-11d0d843330d None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:53.726345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb982285-f765-4233-b98a-11d0d843330d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:53.726631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb982285-f765-4233-b98a-11d0d843330d None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:53.730322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 869/1739] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:45:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:45:53.773986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058762d-4e0e-4fbd-b2e6-d44f97a0bb34 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:53.782628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:53.783211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:53.783493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:53.788238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:53.789172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:53.791699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:53.836213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:53.836566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:53.836758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:53.880595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 871/1740] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:45:53 2022] POST /identity/v3/projects => generated 365 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:53.890211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:53.894821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:53.898980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:54.038296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:54.122328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ded6f-4ef8-49eb-9166-eb2aeaf3e7ee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:54.156881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:54.158378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:54.160765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:54.161389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:54.162738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:54.174532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:54.228167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 870/1741] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:45:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:54.265971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 872/1742] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:45:53 2022] POST /identity/v3/users => generated 289 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:45:54.270230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:54.302033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad089d01-81d2-4856-a815-5bf608d47f2a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:54.317636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:54.318430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:54.318809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:54.319781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:54.320564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:54.322323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:54.347820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 871/1743] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:45:54.352851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:54.370551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01fbf0e-f500-4ab3-a775-0f68a66213e6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:54.388875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:54.390306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:54.391281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:54.393530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] PATH_INFO: `/v3/projects/242c20a676ae495ca8081506a8e4e1e6/users/a5ac9f46422341588effc2bd010b077f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:54.441227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=242c20a676ae495ca8081506a8e4e1e6, user_id=a5ac9f46422341588effc2bd010b077f, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:54.448692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:54.493830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 873/1744] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:54 2022] PUT /identity/v3/projects/242c20a676ae495ca8081506a8e4e1e6/users/a5ac9f46422341588effc2bd010b077f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:54.498054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:54.525772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53fd48-bef4-4aa8-8a6f-e1f0d3639f07 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:54.694214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:54.697645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:54.735883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:54.763696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:54.764665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:54.765621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:54.766567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:54.787111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:54.789690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:54.808931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 872/1745] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:45:54.809988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:54.812138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:54.984665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc9d17-2dfa-49db-8e3b-e24f8b8b0dce admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:55.007008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:55.007736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:55.008037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:55.008381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] PATH_INFO: `/v3/users/394f6f90975d4fe3bf5123182979288c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:55.009709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=394f6f90975d4fe3bf5123182979288c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:55.013345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:55.095265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef58551-5bd5-4db5-b0c8-e03422f0fb2c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:55.136102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:55.143491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:55.149493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:55.150660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:55.171648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '394f6f90975d4fe3bf5123182979288c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.191345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '394f6f90975d4fe3bf5123182979288c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.236435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '394f6f90975d4fe3bf5123182979288c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.239395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '394f6f90975d4fe3bf5123182979288c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.265345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '394f6f90975d4fe3bf5123182979288c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.268724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:55.271656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:55.309492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 874/1746] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:54 2022] DELETE /identity/v3/users/394f6f90975d4fe3bf5123182979288c => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:55.309492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:55.322884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:55.338785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 873/1747] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:55.353473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:55.355034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:55.408782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d7ad7-0122-4480-b19d-916c69050f09 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:55.439618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:55.495598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5a2ac-48dd-4a40-a02a-6b3c1d43f94c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:55.514258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430b9e4-17af-4ccb-9f96-dc9f6e35caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:55.516405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430b9e4-17af-4ccb-9f96-dc9f6e35caca admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:55.517899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430b9e4-17af-4ccb-9f96-dc9f6e35caca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:55.518934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430b9e4-17af-4ccb-9f96-dc9f6e35caca admin admin] PATH_INFO: `/v3/projects/242c20a676ae495ca8081506a8e4e1e6/users/a5ac9f46422341588effc2bd010b077f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:55.533000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430b9e4-17af-4ccb-9f96-dc9f6e35caca admin admin] RBAC: Authorizing `identity:create_grant(project_id=242c20a676ae495ca8081506a8e4e1e6, user_id=a5ac9f46422341588effc2bd010b077f, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:55.536383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430b9e4-17af-4ccb-9f96-dc9f6e35caca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:55.554368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:55.556346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:55.557226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:55.557531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] PATH_INFO: `/v3/projects/445369342220483b9a18e7cc39ae5651` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:55.560597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 875/1748] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:45:55 2022] PUT /identity/v3/projects/242c20a676ae495ca8081506a8e4e1e6/users/a5ac9f46422341588effc2bd010b077f/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:55.561313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] RBAC: Authorizing `identity:delete_project(project_id=445369342220483b9a18e7cc39ae5651)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:55.564740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:55.572370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:55.572690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:55.572915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:55.691792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] MFA Rules not processed for user `a5ac9f46422341588effc2bd010b077f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:55.693588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:55.694902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445369342220483b9a18e7cc39ae5651', 'request_id': 'req-f1115afa-317e-48ad-9d01-d4d0bde8f27a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.752641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445369342220483b9a18e7cc39ae5651', 'request_id': 'req-f1115afa-317e-48ad-9d01-d4d0bde8f27a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.774906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445369342220483b9a18e7cc39ae5651', 'request_id': 'req-f1115afa-317e-48ad-9d01-d4d0bde8f27a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.797035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:55.808634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] The token cache is being invalidate because project 445369342220483b9a18e7cc39ae5651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:55.810087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.811854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:55.814491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 874/1749] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:55 2022] DELETE /identity/v3/projects/445369342220483b9a18e7cc39ae5651 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:55.821323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:55.822947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:55.954464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 876/1750] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:45:55.971032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:55.973664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:56.057528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1115afa-317e-48ad-9d01-d4d0bde8f27a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:56.088380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:56.089065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:56.089345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:56.089725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] PATH_INFO: `/v3/users/b2c0b3ae6d9b40338421e61069bc664a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:56.090875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c0b3ae6d9b40338421e61069bc664a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:56.100630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:56.179674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d52e1-7f64-407d-8aa2-ada84ea01541 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:56.198204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:56.200282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:56.201672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:56.202878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:56.234675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:56.236685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:56.237257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c0b3ae6d9b40338421e61069bc664a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:56.259695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c0b3ae6d9b40338421e61069bc664a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:56.281720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c0b3ae6d9b40338421e61069bc664a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:56.283962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c0b3ae6d9b40338421e61069bc664a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:56.291097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 877/1751] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:56.356850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c0b3ae6d9b40338421e61069bc664a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:56.379947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 875/1752] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:55 2022] DELETE /identity/v3/users/b2c0b3ae6d9b40338421e61069bc664a => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:57.278114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:57.281125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:57.309121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a38981-b24c-4933-abcc-88a51fa55857 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:57.452528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:57.494029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:57.494731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:57.496287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:57.496566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] PATH_INFO: `/v3/projects/195c2ade17064592acc581ed6d3fedba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:57.499607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195c2ade17064592acc581ed6d3fedba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:57.505682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:57.688570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195c2ade17064592acc581ed6d3fedba', 'request_id': 'req-691390ed-1191-49bb-aa7f-dd9eb707dde7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:57.705098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195c2ade17064592acc581ed6d3fedba', 'request_id': 'req-691390ed-1191-49bb-aa7f-dd9eb707dde7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:57.718751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195c2ade17064592acc581ed6d3fedba', 'request_id': 'req-691390ed-1191-49bb-aa7f-dd9eb707dde7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:57.753525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] The token cache is being invalidate because project 195c2ade17064592acc581ed6d3fedba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:57.753739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:57.756141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:57.765125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 878/1753] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:57 2022] DELETE /identity/v3/projects/195c2ade17064592acc581ed6d3fedba => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:57.765621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:57.768654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:57.782503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:57.784233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:57.797955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85694849-1e87-4824-9547-8eb7ff667d79 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:57.984526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691390ed-1191-49bb-aa7f-dd9eb707dde7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:58.001159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:58.001760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:58.002090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:58.002369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] PATH_INFO: `/v3/users/1e5753560e9145bf8e922a422e9cc0da` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:58.004095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5753560e9145bf8e922a422e9cc0da)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:58.006456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:58.062329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:58.065489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:58.065978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:58.067929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] PATH_INFO: `/v3/users/bc1e2b36dc1c4424bd0b78f6468be077` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:58.089073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1e2b36dc1c4424bd0b78f6468be077)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:58.091729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:58.120452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5753560e9145bf8e922a422e9cc0da'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.131781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5753560e9145bf8e922a422e9cc0da'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.158523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5753560e9145bf8e922a422e9cc0da'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.161417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5753560e9145bf8e922a422e9cc0da'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.185598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5753560e9145bf8e922a422e9cc0da'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.188652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:58.191687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:58.206342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 879/1754] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:57 2022] DELETE /identity/v3/users/1e5753560e9145bf8e922a422e9cc0da => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:58.211038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:58.223549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:58.338143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1e2b36dc1c4424bd0b78f6468be077'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.360762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1e2b36dc1c4424bd0b78f6468be077'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.392460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1e2b36dc1c4424bd0b78f6468be077'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.393786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1e2b36dc1c4424bd0b78f6468be077'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.417850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be7ab4-3429-449d-98e2-6dcb7b75f10e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:58.435363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:58.437342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:58.437573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:58.437934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] PATH_INFO: `/v3/projects/670258ffb4334ea6b3ca1c32bb09abfe` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:58.443071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] RBAC: Authorizing `identity:delete_project(project_id=670258ffb4334ea6b3ca1c32bb09abfe)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:58.457921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:58.472977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5681bb89-ad11-41c6-9ac4-64bb6da8c1de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1e2b36dc1c4424bd0b78f6468be077'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.502752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 876/1755] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:45:57 2022] DELETE /identity/v3/users/bc1e2b36dc1c4424bd0b78f6468be077 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:58.504329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:58.506966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:58.507361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:58.578897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:58.597055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:58.693681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670258ffb4334ea6b3ca1c32bb09abfe', 'request_id': 'req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.727836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670258ffb4334ea6b3ca1c32bb09abfe', 'request_id': 'req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.769076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670258ffb4334ea6b3ca1c32bb09abfe', 'request_id': 'req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.809772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 877/1756] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:58.811940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:58.815930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:58.817421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] The token cache is being invalidate because project 670258ffb4334ea6b3ca1c32bb09abfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:58.820478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.821967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4cfaea3c-02c2-4464-9d9a-102f2c28c68e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:58.828210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 880/1757] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:58 2022] DELETE /identity/v3/projects/670258ffb4334ea6b3ca1c32bb09abfe => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:58.833430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:58.833720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:58.833996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:58.850221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2358a-bb80-479f-9733-1681fcd37d13 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:58.901523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:58.903780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.040369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:59.040678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.040759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.041478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] PATH_INFO: `/v3/projects/cba686e736b64257a940b10b0b7e3121` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.051777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba686e736b64257a940b10b0b7e3121)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:59.053903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:59.177635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 881/1758] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:59.179115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:59.182859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.257794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:59.274649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba686e736b64257a940b10b0b7e3121', 'request_id': 'req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.285329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba686e736b64257a940b10b0b7e3121', 'request_id': 'req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.288370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:59.315962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba686e736b64257a940b10b0b7e3121', 'request_id': 'req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.342631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] The token cache is being invalidate because project cba686e736b64257a940b10b0b7e3121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:59.342964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.344436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3e746fe2-d61d-46d6-a2ae-e2dbc0462520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.350672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 878/1759] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:58 2022] DELETE /identity/v3/projects/cba686e736b64257a940b10b0b7e3121 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:59.360854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.361159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.361439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.365399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf10318-cb94-466e-80a6-9038b9db5a96 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:59.415706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] MFA Rules not processed for user `cdb5ce192c764fe4946542f5960ccd39`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:59.417121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.514948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:59.518767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.521082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.522439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.534766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.556302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 879/1760] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:59.556302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:59.556302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.693495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:59.695698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8fa71-0452-4975-902d-1b66b42a1b3d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:59.710873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 882/1761] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:45:59.712454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8486-1757-4f13-a8e7-e23a3c37e08e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:59.712736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4feb-7154-4e27-8fb7-c72e32104969 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.713091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4feb-7154-4e27-8fb7-c72e32104969 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.713437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f4feb-7154-4e27-8fb7-c72e32104969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.719550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:59.720425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.720821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.721155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] PATH_INFO: `/v3/projects/85f64e75af344a9e9657becff128eedc` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.722838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85f64e75af344a9e9657becff128eedc)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:59.729991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:59.754596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a30f4feb-7154-4e27-8fb7-c72e32104969 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:59.756679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a30f4feb-7154-4e27-8fb7-c72e32104969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.767424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 883/1762] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:59.769255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.769778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.770574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.800870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:59.802425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.814818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 884/1763] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:45:59.821807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:45:59.842267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85f64e75af344a9e9657becff128eedc', 'request_id': 'req-f5176877-42a6-4b44-b1cb-3ff673e2f48c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.851253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85f64e75af344a9e9657becff128eedc', 'request_id': 'req-f5176877-42a6-4b44-b1cb-3ff673e2f48c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.855123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e1a80-83ae-45d1-9b42-cd188b273c66 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:45:59.864837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85f64e75af344a9e9657becff128eedc', 'request_id': 'req-f5176877-42a6-4b44-b1cb-3ff673e2f48c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.894263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] The token cache is being invalidate because project 85f64e75af344a9e9657becff128eedc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:45:59.894566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.896541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f5176877-42a6-4b44-b1cb-3ff673e2f48c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:45:59.899513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 880/1764] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:45:59 2022] DELETE /identity/v3/projects/85f64e75af344a9e9657becff128eedc => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:45:59.904055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.904443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.904840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.930895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:45:59.952233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:45:59.953072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.953363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.953633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:45:59.955011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:45:59.955315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:45:59.956983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c617a-8c5c-40e3-b5ee-d8d103f58868 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:45:59.966465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:45:59.969606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:45:59.980644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 885/1765] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:45:59.980644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:45:59.981328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:45:59.981941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.006395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 881/1766] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:00.007382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.013848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.036681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15355838-f759-48d5-9f89-0c75c4926792 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.044283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879deafc-a304-4988-a46a-5c57d08f95a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.045576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:00.046684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879deafc-a304-4988-a46a-5c57d08f95a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.046978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.047279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879deafc-a304-4988-a46a-5c57d08f95a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.047890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879deafc-a304-4988-a46a-5c57d08f95a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.048783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879deafc-a304-4988-a46a-5c57d08f95a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.054731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879deafc-a304-4988-a46a-5c57d08f95a1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.105628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 886/1767] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:00.113537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.122152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 882/1768] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:00.125111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f535486-08f8-4f6c-bc7c-4fec5e3094dd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.132377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:00.134175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0fb7-f133-4412-8353-e98bc5010b7f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.134783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0fb7-f133-4412-8353-e98bc5010b7f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.135883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0fb7-f133-4412-8353-e98bc5010b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.180070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dd4d0fb7-f133-4412-8353-e98bc5010b7f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:00.181364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dd4d0fb7-f133-4412-8353-e98bc5010b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.253159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.253873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.254472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.254828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.255725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:00.256124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.258264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.296671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 887/1769] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:00.297927 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.313131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 883/1770] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:00.314943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f2ab-b788-4ce2-a2d1-08bf1db24c14 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.317757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f2ab-b788-4ce2-a2d1-08bf1db24c14 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.318243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f2ab-b788-4ce2-a2d1-08bf1db24c14 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.321377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e47f3-bbaa-4329-9720-a79cd10e5473 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.325999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 884/1771] 10.222.0.52 () {54 vars in 912 bytes} [Fri Oct 28 09:46:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:46:00.327460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f2ab-b788-4ce2-a2d1-08bf1db24c14 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.343930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.346392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.346622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.347128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.347841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.349775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f2ab-b788-4ce2-a2d1-08bf1db24c14 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.349914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.358158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eced816-c99f-45e5-acde-7d961321b35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.359693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eced816-c99f-45e5-acde-7d961321b35f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.360047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eced816-c99f-45e5-acde-7d961321b35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.360312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eced816-c99f-45e5-acde-7d961321b35f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.361109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eced816-c99f-45e5-acde-7d961321b35f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.363382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eced816-c99f-45e5-acde-7d961321b35f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.383155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 888/1772] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/users => generated 294 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:00.384808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.399594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6297470-4252-4a84-b22e-f431165f9781 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.405484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.406500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.407053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.407531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.411788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 885/1773] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:00.411973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:00.411973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.412139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.412139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.412139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.412406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.427484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 889/1774] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:00.428983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.443627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea8fa0-dbf4-4ecf-aafa-0584d4c7975a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.444407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] MFA Rules not processed for user `b07b6ea0a9a7475fbcc8d6baf537cc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:00.445749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.523160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.523830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.524092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.524482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.525285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.527078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.536313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 886/1775] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:00.537488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.541572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 890/1776] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:00.541572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.552785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743dfad-2ee8-4bf2-9033-b00687b38908 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.553459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b06a45-7caf-4475-bece-b840a350c5d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.557990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.558815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.559057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.559263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.559553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.559771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.559979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.560256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.560630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.560895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.562834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.563263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.596579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 891/1777] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:00.596807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 887/1778] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/users => generated 286 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:00.596807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.599303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.601335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.617783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2915b6-675c-47ce-b9d7-41a8790fb152 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.659684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:00.681088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:00.684435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9005f-5e18-497a-8fde-098e457992f5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.691479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.692275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.692621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.692900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] PATH_INFO: `/v3/projects/8fa541e8887844b3a5bccbfa866f63a8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.694765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa541e8887844b3a5bccbfa866f63a8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.696821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.702751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.703638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.704454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.704801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.705463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.707359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.720337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 888/1779] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:00.721242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.723155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.763379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa541e8887844b3a5bccbfa866f63a8', 'request_id': 'req-d495f0f3-f878-4ef7-9dba-fb673b124eeb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:00.776526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa541e8887844b3a5bccbfa866f63a8', 'request_id': 'req-d495f0f3-f878-4ef7-9dba-fb673b124eeb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:00.790148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa541e8887844b3a5bccbfa866f63a8', 'request_id': 'req-d495f0f3-f878-4ef7-9dba-fb673b124eeb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:00.801011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37d505e-23dd-43e7-8ba9-a66a0f8e4770 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.810795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] The token cache is being invalidate because project 8fa541e8887844b3a5bccbfa866f63a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:00.811195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:00.812535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:00.814364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 892/1780] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:00 2022] DELETE /identity/v3/projects/8fa541e8887844b3a5bccbfa866f63a8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:00.815177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.816618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.873106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.873922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.874207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.874481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.876730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.887515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.889415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.893236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d495f0f3-f878-4ef7-9dba-fb673b124eeb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.898365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.898565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 889/1781] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:00.899233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.899486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.899616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.900098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.900883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.901012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.902868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.915996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906edff-aa64-461c-9229-9b04594819c2 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.922196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.922822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.923103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.923400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.924106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.927218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:00.942759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 890/1782] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:00.942759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.943069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 893/1783] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:00 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:00.944976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:00.947892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.948333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:00.964832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b767bf1-1b06-40e9-948d-6db6e9688ef4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.969714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb6f7a-fc3e-4b21-be94-c9425235691e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:00.970765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.971624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.971941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.972216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] PATH_INFO: `/v3/projects/ce8b9b8e7ab04f5d89e3b94fffe8ab02/users/27b71b94dcd2406fbbe339904affd8a5/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.976613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:00.977609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:00.977937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:00.978318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] PATH_INFO: `/v3/users/a5ac9f46422341588effc2bd010b077f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:00.979577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ac9f46422341588effc2bd010b077f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:00.982564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.002613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8b9b8e7ab04f5d89e3b94fffe8ab02, user_id=27b71b94dcd2406fbbe339904affd8a5, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.005328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.024753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 891/1784] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:00 2022] PUT /identity/v3/projects/ce8b9b8e7ab04f5d89e3b94fffe8ab02/users/27b71b94dcd2406fbbe339904affd8a5/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:01.025883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.052681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ac9f46422341588effc2bd010b077f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:01.062787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ac9f46422341588effc2bd010b077f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:01.071112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=79,keystone:DELETE=16,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:01.083602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ac9f46422341588effc2bd010b077f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:01.083835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ac9f46422341588effc2bd010b077f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:01.095915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ac9f46422341588effc2bd010b077f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:01.104727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 894/1785] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:00 2022] DELETE /identity/v3/users/a5ac9f46422341588effc2bd010b077f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:01.109085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.109085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:01.140321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f37156-3e50-42fe-a9af-df1978be02e8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.154478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:01.207215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccd01a-d5d2-427c-8a86-33a622a662cf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.213142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46accd15-aa05-47e8-a591-2f519405b1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.213909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46accd15-aa05-47e8-a591-2f519405b1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.214331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46accd15-aa05-47e8-a591-2f519405b1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.215845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46accd15-aa05-47e8-a591-2f519405b1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.217611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46accd15-aa05-47e8-a591-2f519405b1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.220078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46accd15-aa05-47e8-a591-2f519405b1d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.224258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.225753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.226041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.226315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.226988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.229535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.237632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 895/1786] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:01.239282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3ff9-5c05-4b6d-bd52-b99c15cfc182 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.239663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3ff9-5c05-4b6d-bd52-b99c15cfc182 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.240058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea3ff9-5c05-4b6d-bd52-b99c15cfc182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.248030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 892/1787] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:01.249177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.265239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141f7c4-3bcf-4a82-9a40-349328397ca3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.270712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.271354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.272089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.272442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.276560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.280307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.288457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9aea3ff9-5c05-4b6d-bd52-b99c15cfc182 None None] MFA Rules not processed for user `27b71b94dcd2406fbbe339904affd8a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:01.290041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9aea3ff9-5c05-4b6d-bd52-b99c15cfc182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:01.293119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 893/1788] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:01.294338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.300459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:01.316892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d72c4-1a81-4df3-a7de-d2414071e5a8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.323360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.324332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.324684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.325073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] PATH_INFO: `/v3/projects/233d86e131f84148927ec3ed0f994580/users/9dc6776dfc5d402bb9f5afac5dcce2a6/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.347719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] RBAC: Authorizing `identity:create_grant(project_id=233d86e131f84148927ec3ed0f994580, user_id=9dc6776dfc5d402bb9f5afac5dcce2a6, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.351567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.372746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 894/1789] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:01 2022] PUT /identity/v3/projects/233d86e131f84148927ec3ed0f994580/users/9dc6776dfc5d402bb9f5afac5dcce2a6/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:01.373936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.443732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 896/1790] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:01.445172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7940b-06b0-4251-9dc0-c53c05f0c0d3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.445586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7940b-06b0-4251-9dc0-c53c05f0c0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.445933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7940b-06b0-4251-9dc0-c53c05f0c0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.511812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6ce7940b-06b0-4251-9dc0-c53c05f0c0d3 None None] MFA Rules not processed for user `9dc6776dfc5d402bb9f5afac5dcce2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:01.519673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae70636-6afd-43da-bbe2-789576ee24be admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.524308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ce7940b-06b0-4251-9dc0-c53c05f0c0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:01.544867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.545777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.546353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.546858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] PATH_INFO: `/v3/projects/d9ffbf572d704008b96c92031d69220a/users/56f11eef6105424f9b0783c56bd0a3d7/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.599437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ffbf572d704008b96c92031d69220a, user_id=56f11eef6105424f9b0783c56bd0a3d7, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.603964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.639111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 895/1791] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:01 2022] PUT /identity/v3/projects/d9ffbf572d704008b96c92031d69220a/users/56f11eef6105424f9b0783c56bd0a3d7/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:01.640291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.660282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:01.661625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:01.697716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 897/1792] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:01.700496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.701180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.701503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.757745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] MFA Rules not processed for user `56f11eef6105424f9b0783c56bd0a3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:01.759461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:01.843946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:01.858085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071c820-8355-4dad-89a2-d944f1540fc1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.888460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.889274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.890311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 898/1793] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:01.892287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.892762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.893736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.894838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.911611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.928943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d415a3-4c10-4d17-a5c4-3f841e61d7b1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:01.938219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:01.939259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:01.939655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:01.940091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:01.940987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:01.946586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:01.964858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 896/1794] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:01 2022] POST /identity/v3/projects => generated 387 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:01.966166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:01.970707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:02.052752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 899/1795] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:01 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:02.056113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.103310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc3009-c5f3-4cbe-b3dd-beabd6fcafc2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.160475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc57b6-858c-40c8-a674-186f1634b6d1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.166222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.166985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.167326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.167701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.168515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.172848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.210381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.211411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.211860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.212244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] PATH_INFO: `/v3/projects/242c20a676ae495ca8081506a8e4e1e6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.213795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242c20a676ae495ca8081506a8e4e1e6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.215590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 900/1796] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:02.216700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.220636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.283251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:02.288585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242c20a676ae495ca8081506a8e4e1e6', 'request_id': 'req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:02.295161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242c20a676ae495ca8081506a8e4e1e6', 'request_id': 'req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:02.299465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6906fbd-68f8-490b-9333-c49144c0a7d3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.302102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242c20a676ae495ca8081506a8e4e1e6', 'request_id': 'req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:02.318591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] The token cache is being invalidate because project 242c20a676ae495ca8081506a8e4e1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:02.319789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:02.320624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:02.322214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 897/1797] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:01 2022] DELETE /identity/v3/projects/242c20a676ae495ca8081506a8e4e1e6 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:02.322990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.324334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:02.347789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:02.362855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.363515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.363804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.364083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.364771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.367420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.391374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 901/1798] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/users => generated 300 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:02.392236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.397067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:02.417470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cb1fd-8b0c-4b8b-a6b7-bc35f8a7b8f6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.425501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51e25-327f-416d-b8da-1777a6beeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.425863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2ed34-f7df-4381-a0bf-0670d5a160b8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.426437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51e25-327f-416d-b8da-1777a6beeef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.426890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51e25-327f-416d-b8da-1777a6beeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.427303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51e25-327f-416d-b8da-1777a6beeef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.429092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd51e25-327f-416d-b8da-1777a6beeef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.433457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd51e25-327f-416d-b8da-1777a6beeef0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.434450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.435404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.435890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.436331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.437361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.440529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.465151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 902/1799] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/users => generated 300 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:02.469379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.469649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.469915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.473027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 898/1800] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:02.474053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.476510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:02.490324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f7bbb-4227-452f-b6e6-92249e1c36e6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.492878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:02.494072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:02.495773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.496388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.496665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.496939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.497585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.499172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.510410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 899/1801] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:02.511283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.516316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 903/1802] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:02.516316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.527225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619be1-297a-40b3-87ea-eeb77b5f2be0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.528675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85b230-efc6-4933-a602-4df4c174ab83 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.531944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.532553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.532822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.533089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.533472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6f2d3-f6d6-45af-8292-355a5abf9fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.533729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.534099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6f2d3-f6d6-45af-8292-355a5abf9fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.534380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6f2d3-f6d6-45af-8292-355a5abf9fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.534681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6f2d3-f6d6-45af-8292-355a5abf9fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.535335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6f2d3-f6d6-45af-8292-355a5abf9fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.535487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.536951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6f2d3-f6d6-45af-8292-355a5abf9fd0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.545851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 904/1803] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:02.546642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.549396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 900/1804] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:02.549814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.550102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.550375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.560422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379c7d7-e6b8-4208-ad49-0c20f8e1526f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.566935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.567681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.568524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.568974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] PATH_INFO: `/v3/projects/1e3071cadac64b1ba0b03a41e040c491/users/1f33b633540649ab947c1ae69a76a89a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.576445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:02.577031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:02.587709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3071cadac64b1ba0b03a41e040c491, user_id=1f33b633540649ab947c1ae69a76a89a, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.588086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 901/1805] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:02.589262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.594403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.602694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f977a3-5cdc-4089-b441-081790cb43ab None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.608125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.608799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.609080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.609346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] PATH_INFO: `/v3/projects/603f3324235e4e7aac2e5abdcc297066/users/67633cf5c142472d9e47291f6e11ad0d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.612685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 905/1806] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:02 2022] PUT /identity/v3/projects/1e3071cadac64b1ba0b03a41e040c491/users/1f33b633540649ab947c1ae69a76a89a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:02.613614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.629437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603f3324235e4e7aac2e5abdcc297066, user_id=67633cf5c142472d9e47291f6e11ad0d, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.633075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.650704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 902/1807] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:02 2022] PUT /identity/v3/projects/603f3324235e4e7aac2e5abdcc297066/users/67633cf5c142472d9e47291f6e11ad0d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:02.651537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.698745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b875129-93eb-4a84-98db-f29b8840d4ab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.712580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922be60c-e429-4ebf-a6e8-9dcea7879173 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.717225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.717827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.718100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.718366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.718998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:02.719291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.720886 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.727808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:02.738303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 903/1808] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:02.738303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.748217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b10f35-9ae6-438f-8e75-1b669cc3979b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.753003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.753619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.753887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.754151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.754786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.756963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.762020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.762714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.762989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.763261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] PATH_INFO: `/v3/projects/f1f87e29338d46e6a37b63aebf01ecc0/users/59b4377c82e244c28ab96e62f966c31a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.769054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 904/1809] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:02.769926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.781289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455aa12e-6cc7-4ae6-9548-f5374f001b55 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.782509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f87e29338d46e6a37b63aebf01ecc0, user_id=59b4377c82e244c28ab96e62f966c31a, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.785267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.786109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.786728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.787004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.787298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.788077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.789782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.799616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 906/1810] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:02 2022] PUT /identity/v3/projects/f1f87e29338d46e6a37b63aebf01ecc0/users/59b4377c82e244c28ab96e62f966c31a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:02.800683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.803900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 905/1811] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:02.803900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.812603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02abae-fd2b-4ad2-abd1-ecc5ebee9984 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.839639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:02.875471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ac351-760b-4051-85cd-e7ef2101882a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:02.885419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.886196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.886567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.886918 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.887421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:02.887675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.888618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:02.889512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:02.889825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] PATH_INFO: `/v3/projects/1e3071cadac64b1ba0b03a41e040c491/users/1f33b633540649ab947c1ae69a76a89a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:02.890372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.891989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3071cadac64b1ba0b03a41e040c491, user_id=1f33b633540649ab947c1ae69a76a89a, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:02.894766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:02.907071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 906/1812] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:02 2022] PUT /identity/v3/projects/1e3071cadac64b1ba0b03a41e040c491/users/1f33b633540649ab947c1ae69a76a89a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:02.917361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:02.927800 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 907/1813] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:02 2022] POST /identity/v3/projects => generated 355 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:02.927800 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.008028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaf9c6-e65d-459d-8749-f69370a3b6fa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.008235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a51a-6ea2-4f4e-b52c-f816d8948200 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.016004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.016831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.017000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dded266-8fc2-4621-8486-be7cf798ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.017290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.017691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] PATH_INFO: `/v3/projects/603f3324235e4e7aac2e5abdcc297066/users/67633cf5c142472d9e47291f6e11ad0d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.018208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dded266-8fc2-4621-8486-be7cf798ddca admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.018758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dded266-8fc2-4621-8486-be7cf798ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.019372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dded266-8fc2-4621-8486-be7cf798ddca admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.020319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603f3324235e4e7aac2e5abdcc297066, user_id=67633cf5c142472d9e47291f6e11ad0d, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.020659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dded266-8fc2-4621-8486-be7cf798ddca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.023096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dded266-8fc2-4621-8486-be7cf798ddca admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.024892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.036228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 907/1814] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:03.037674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.038027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.038368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.042397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 908/1815] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:02 2022] PUT /identity/v3/projects/603f3324235e4e7aac2e5abdcc297066/users/67633cf5c142472d9e47291f6e11ad0d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:03.043408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.073995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] MFA Rules not processed for user `1f33b633540649ab947c1ae69a76a89a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:03.075413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:03.104501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:03.159530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e0d90c-21a7-458d-ae11-4230e2d98cf3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.167003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08998a-ae2c-4a01-a05f-443c2a391caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.167960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08998a-ae2c-4a01-a05f-443c2a391caa admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.168249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08998a-ae2c-4a01-a05f-443c2a391caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.168707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08998a-ae2c-4a01-a05f-443c2a391caa admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.170205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08998a-ae2c-4a01-a05f-443c2a391caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.173745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08998a-ae2c-4a01-a05f-443c2a391caa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.203274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 908/1816] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:03.204522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.212235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 909/1817] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:03 2022] POST /identity/v3/users => generated 278 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:03.213393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.213686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.214580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.229953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56e479-8b1a-48f7-9a78-861c01cb227c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.245895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.246594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.246877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.247331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] PATH_INFO: `/v3/projects/f1f87e29338d46e6a37b63aebf01ecc0/users/59b4377c82e244c28ab96e62f966c31a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.249774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1f87e29338d46e6a37b63aebf01ecc0, user_id=59b4377c82e244c28ab96e62f966c31a, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.253343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.254309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] MFA Rules not processed for user `67633cf5c142472d9e47291f6e11ad0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:03.256023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:03.271419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 909/1818] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:03 2022] PUT /identity/v3/projects/f1f87e29338d46e6a37b63aebf01ecc0/users/59b4377c82e244c28ab96e62f966c31a/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:03.273314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.329451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:03.347913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 910/1819] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:03.353139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.355136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:03.376729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e2b2d-ce71-4202-8eaf-011db73d4dd1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.383086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103b6a9-be5c-49cc-b476-117d6e4d95a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.383868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103b6a9-be5c-49cc-b476-117d6e4d95a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.384604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103b6a9-be5c-49cc-b476-117d6e4d95a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.384957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103b6a9-be5c-49cc-b476-117d6e4d95a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.385756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103b6a9-be5c-49cc-b476-117d6e4d95a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.392875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103b6a9-be5c-49cc-b476-117d6e4d95a7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.407788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 910/1820] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:03.407788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.407788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.407788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.439608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb2e86-af1f-466b-8cf8-3a04c17b61c2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.442350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] MFA Rules not processed for user `59b4377c82e244c28ab96e62f966c31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:03.443661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:03.502767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.503396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.503687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.503980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.538184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:03.545565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 911/1821] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:03.545565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.558821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a266fd44-9003-4dd5-960c-0d92dee41cef None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.565809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.566610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.567226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.567670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.568673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.571429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.592311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 912/1822] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:03.593425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.605117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48d9b-91ec-4cd6-a999-78b0086473fa admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.609086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.610714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.611730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.612921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.614141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.614494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] PATH_INFO: `/v3/projects/3f4b446b18d148638b974bab990679c8/users/5eba40ff99644b54a344044be1135d70/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.623583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 911/1823] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:03.640031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f4b446b18d148638b974bab990679c8, user_id=5eba40ff99644b54a344044be1135d70, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.645644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.659785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.667845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 913/1824] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:03 2022] PUT /identity/v3/projects/3f4b446b18d148638b974bab990679c8/users/5eba40ff99644b54a344044be1135d70/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:03.677208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.688467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f660c-5c0b-4c3d-ab71-70f2ef61d42d service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.783846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d207e-f756-46ec-a630-f81763ff2808 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:03.789124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ca231f-6391-4d49-98af-dba934ddb038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.789827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca231f-6391-4d49-98af-dba934ddb038 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.790111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca231f-6391-4d49-98af-dba934ddb038 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.790383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ca231f-6391-4d49-98af-dba934ddb038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.803094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:03.804663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.805024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.805393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] PATH_INFO: `/v3/users/27b71b94dcd2406fbbe339904affd8a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.806803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b71b94dcd2406fbbe339904affd8a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.809499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.853567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:03.899958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b71b94dcd2406fbbe339904affd8a5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:03.906146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca231f-6391-4d49-98af-dba934ddb038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:03.908134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ca231f-6391-4d49-98af-dba934ddb038 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:03.910100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b71b94dcd2406fbbe339904affd8a5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:03.920065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 912/1825] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:03.920609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b71b94dcd2406fbbe339904affd8a5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:03.921411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b6a60-c501-4c85-a43a-ce7b7adfc42e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:03.921645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b6a60-c501-4c85-a43a-ce7b7adfc42e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:03.921805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b71b94dcd2406fbbe339904affd8a5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:03.921942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b6a60-c501-4c85-a43a-ce7b7adfc42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:03.941699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b71b94dcd2406fbbe339904affd8a5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:03.953904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 914/1826] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:03 2022] DELETE /identity/v3/users/27b71b94dcd2406fbbe339904affd8a5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:03.953904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:03.953904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:03.979042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d71b6a60-c501-4c85-a43a-ce7b7adfc42e None None] MFA Rules not processed for user `5eba40ff99644b54a344044be1135d70`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:03.981826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d71b6a60-c501-4c85-a43a-ce7b7adfc42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.036184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905000e-f3ba-4258-90c9-a2e2cd286cb1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:04.044228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:04.045223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:04.045532 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:04.045914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:04.048153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.089998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:04.114133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 913/1827] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:04.117588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:04.118066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:04.118543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:04.154627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:04.160709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.210405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:04.213125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:04.233361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 915/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:04.234612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:04.237062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.342126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:04.356012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 914/1829] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:04.383195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:04.386949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.411124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a49e-6f0f-4e8f-b375-d89c4964d178 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:04.423121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:04.424409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:04.424814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:04.425192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:04.426202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:04.434136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:04.496777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 916/1830] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:04 2022] POST /identity/v3/projects => generated 357 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:04.501063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:04.503711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.603961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692c567-7484-4468-ae28-2cecbaf2be9f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:04.723478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c33ee-66e3-44ba-ac0a-45552b94e77e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:04.732885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:04.734456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:04.734850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:04.735196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:04.738785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:04.739960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:04.740450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:04.741548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] PATH_INFO: `/v3/users/56f11eef6105424f9b0783c56bd0a3d7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:04.743942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f11eef6105424f9b0783c56bd0a3d7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:04.752323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:04.777087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:04.781878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:04.798793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 915/1831] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:04.799694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:04.836773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074c5bd-946f-4c7e-afcd-410cfcb4e8d8 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:04.844869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:04.845823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:04.846536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:04.847110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:04.848531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:04.852801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:04.860244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f11eef6105424f9b0783c56bd0a3d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:04.877873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f11eef6105424f9b0783c56bd0a3d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:04.891281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f11eef6105424f9b0783c56bd0a3d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:04.892096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f11eef6105424f9b0783c56bd0a3d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:04.907947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 916/1832] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:04 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:04.909470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:04.914586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.923032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f11eef6105424f9b0783c56bd0a3d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:04.933311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 917/1833] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:04 2022] DELETE /identity/v3/users/56f11eef6105424f9b0783c56bd0a3d7 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:04.934358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:04.936030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:04.949345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c120e-d4db-4bbc-abeb-dccc39daf2b9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.009127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:05.025495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:05.043079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.044149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.044768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.045225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] PATH_INFO: `/v3/users/9dc6776dfc5d402bb9f5afac5dcce2a6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.046779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc6776dfc5d402bb9f5afac5dcce2a6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.049110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.077190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351c9ad-02a6-472c-bf3d-b224fe7d57ab admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.090125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd8ef-c0f7-423f-bde7-48cdec062215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.091389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd8ef-c0f7-423f-bde7-48cdec062215 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.092215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd8ef-c0f7-423f-bde7-48cdec062215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.092845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd8ef-c0f7-423f-bde7-48cdec062215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.094156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bd8ef-c0f7-423f-bde7-48cdec062215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.097302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bd8ef-c0f7-423f-bde7-48cdec062215 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.156321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc6776dfc5d402bb9f5afac5dcce2a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:05.164265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 918/1834] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:05.166166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.166437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.181674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc6776dfc5d402bb9f5afac5dcce2a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:05.183447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.223230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc6776dfc5d402bb9f5afac5dcce2a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:05.225028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc6776dfc5d402bb9f5afac5dcce2a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:05.238498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:05.240041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:05.247207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745b7173-8fbb-4d7a-8f03-4902e2a6cecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc6776dfc5d402bb9f5afac5dcce2a6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:05.251993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 919/1835] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:05.253114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:05.254516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:05.263856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 917/1836] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:04 2022] DELETE /identity/v3/users/9dc6776dfc5d402bb9f5afac5dcce2a6 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:05.270307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.271508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.271888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.274139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7358bd2-b997-4d8a-9041-cb00d52b97be None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.329331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:05.330735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:05.448982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.453007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.453342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.453921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] PATH_INFO: `/v3/projects/c067a61dd8744757a5aa348e702c19ae/users/8a80f41069d449b0ae736a19c214b702/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.499489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c067a61dd8744757a5aa348e702c19ae, user_id=8a80f41069d449b0ae736a19c214b702, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.508794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 918/1837] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:05.509223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.509932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:05.516007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:05.535995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 920/1838] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:05 2022] PUT /identity/v3/projects/c067a61dd8744757a5aa348e702c19ae/users/8a80f41069d449b0ae736a19c214b702/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:05.541805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:05.639805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a960a2-3fad-464e-9638-01f874357098 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.645086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.645770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.646043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.646316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.647018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.648309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c712-283e-4caf-aae0-00342304ab2e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.649386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.660618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:05.661663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 921/1839] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:05.662771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:05.664991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:05.728338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.728989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.729270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.729552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.747935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.756291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.798503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 919/1840] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:05.799616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:05.811528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbc8f5-ffc0-4d31-b111-abce48097806 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.818559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cce27-8be6-436f-aaab-ed09cf30d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.821012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cce27-8be6-436f-aaab-ed09cf30d1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.821852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a051082-4a37-4f98-b515-d85a9217a3a2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.824080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cce27-8be6-436f-aaab-ed09cf30d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.824550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cce27-8be6-436f-aaab-ed09cf30d1cd admin admin] PATH_INFO: `/v3/projects/c067a61dd8744757a5aa348e702c19ae/users/8a80f41069d449b0ae736a19c214b702/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.826572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:05.827371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.827789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.828161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.829924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cce27-8be6-436f-aaab-ed09cf30d1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c067a61dd8744757a5aa348e702c19ae, user_id=8a80f41069d449b0ae736a19c214b702, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.837861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cce27-8be6-436f-aaab-ed09cf30d1cd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.842190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:05.846084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:05.856199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:05.874497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 922/1842] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:05.874751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 920/1842] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:05 2022] PUT /identity/v3/projects/c067a61dd8744757a5aa348e702c19ae/users/8a80f41069d449b0ae736a19c214b702/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:05.876030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:05.880114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:05.882502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:05.882814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:05.883075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:05.924084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180790-5698-4abc-8e18-005ebbd4b65c service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:05.963964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] MFA Rules not processed for user `8a80f41069d449b0ae736a19c214b702`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:05.966801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:06.047118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:06.050787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:06.051364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:06.052167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] PATH_INFO: `/v3/projects/ce8b9b8e7ab04f5d89e3b94fffe8ab02` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:06.054896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8b9b8e7ab04f5d89e3b94fffe8ab02)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:06.057842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:06.110899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 921/1843] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:06.138800 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:06.144942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:06.178939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8b9b8e7ab04f5d89e3b94fffe8ab02', 'request_id': 'req-485f6e87-b35a-46d1-8175-b8def917bd72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.193704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8b9b8e7ab04f5d89e3b94fffe8ab02', 'request_id': 'req-485f6e87-b35a-46d1-8175-b8def917bd72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.216977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8b9b8e7ab04f5d89e3b94fffe8ab02', 'request_id': 'req-485f6e87-b35a-46d1-8175-b8def917bd72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.248480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] The token cache is being invalidate because project ce8b9b8e7ab04f5d89e3b94fffe8ab02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:06.249687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.250637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.256269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 923/1844] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:05 2022] DELETE /identity/v3/projects/ce8b9b8e7ab04f5d89e3b94fffe8ab02 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:06.258632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:06.261266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:06.353744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4b711-43be-441e-a4c1-adc5e92b3e85 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:06.434826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f6e87-b35a-46d1-8175-b8def917bd72 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:06.442860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:06.444595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:06.445035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:06.445479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] PATH_INFO: `/v3/users/1f33b633540649ab947c1ae69a76a89a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:06.447547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f33b633540649ab947c1ae69a76a89a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:06.452332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:06.456561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:06.475162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:06.477956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:06.478430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:06.478882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:06.482506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:06.539033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:06.562780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f33b633540649ab947c1ae69a76a89a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.578221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f33b633540649ab947c1ae69a76a89a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.601468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f33b633540649ab947c1ae69a76a89a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.606675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f33b633540649ab947c1ae69a76a89a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.642735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f33b633540649ab947c1ae69a76a89a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:06.643254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:06.647901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:06.656348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 924/1845] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:06 2022] DELETE /identity/v3/users/1f33b633540649ab947c1ae69a76a89a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:06.698924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 922/1846] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:06.880242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:06.883942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:07.020603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b358d34-0139-490e-8d99-4aaaed0822f9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:07.026989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:07.028293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:07.028721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:07.029089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:07.035371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:07.045463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:07.048607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:07.107006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6eefe-e30a-4af8-aa1e-4ceafc327b9b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:07.158332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:07.163413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:07.192406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 925/1847] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:07.278897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:07.279707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:07.281560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:07.283477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] PATH_INFO: `/v3/projects/d9ffbf572d704008b96c92031d69220a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:07.284711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ffbf572d704008b96c92031d69220a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:07.289990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:07.358987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:07.360974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:07.407373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d845cbd-1828-43b8-8bf0-4b06804c700f service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:07.483148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ffbf572d704008b96c92031d69220a', 'request_id': 'req-67501bd5-c838-4b5a-b229-363150af1058'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.501176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ffbf572d704008b96c92031d69220a', 'request_id': 'req-67501bd5-c838-4b5a-b229-363150af1058'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.515003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ffbf572d704008b96c92031d69220a', 'request_id': 'req-67501bd5-c838-4b5a-b229-363150af1058'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.571231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] The token cache is being invalidate because project d9ffbf572d704008b96c92031d69220a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:07.572930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.577815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.588276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 923/1848] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:07 2022] DELETE /identity/v3/projects/d9ffbf572d704008b96c92031d69220a => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:07.589325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:07.601281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:07.641223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:07.642281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:07.642741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:07.643878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] PATH_INFO: `/v3/projects/233d86e131f84148927ec3ed0f994580` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:07.646300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=233d86e131f84148927ec3ed0f994580)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:07.652355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:07.667711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:07.684451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:07.772911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67501bd5-c838-4b5a-b229-363150af1058 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:07.781658 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233d86e131f84148927ec3ed0f994580', 'request_id': 'req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.805486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233d86e131f84148927ec3ed0f994580', 'request_id': 'req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.822494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233d86e131f84148927ec3ed0f994580', 'request_id': 'req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.855880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] The token cache is being invalidate because project 233d86e131f84148927ec3ed0f994580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:07.856592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.857691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:07.860737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 926/1849] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:07 2022] DELETE /identity/v3/projects/233d86e131f84148927ec3ed0f994580 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:07.862740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:07.865800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:07.894422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:07.894676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:07.894925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:07.895648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] PATH_INFO: `/v3/projects/1e3071cadac64b1ba0b03a41e040c491` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:07.897601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3071cadac64b1ba0b03a41e040c491)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:07.899946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:07.971048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9374c68-0c66-44f3-9d41-f2e9c3137e9e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:07.976347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:07.977011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:07.977286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:07.977816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] PATH_INFO: `/v3/users/59b4377c82e244c28ab96e62f966c31a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:07.979420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b4377c82e244c28ab96e62f966c31a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:07.982055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:08.003983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3071cadac64b1ba0b03a41e040c491', 'request_id': 'req-f196dc4e-a86d-4b4d-9de7-231e76809464'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.015339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3071cadac64b1ba0b03a41e040c491', 'request_id': 'req-f196dc4e-a86d-4b4d-9de7-231e76809464'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.023361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3071cadac64b1ba0b03a41e040c491', 'request_id': 'req-f196dc4e-a86d-4b4d-9de7-231e76809464'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.036650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] The token cache is being invalidate because project 1e3071cadac64b1ba0b03a41e040c491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:08.036944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.037657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.039793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 924/1850] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:07 2022] DELETE /identity/v3/projects/1e3071cadac64b1ba0b03a41e040c491 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:08.040462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:08.041884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:08.051913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b4377c82e244c28ab96e62f966c31a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.062065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b4377c82e244c28ab96e62f966c31a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.078666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b4377c82e244c28ab96e62f966c31a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.080223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b4377c82e244c28ab96e62f966c31a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.108369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-abf5a42f-a47c-4dac-b65c-4394f65c7307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b4377c82e244c28ab96e62f966c31a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.120291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 927/1851] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:07 2022] DELETE /identity/v3/users/59b4377c82e244c28ab96e62f966c31a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:08.122489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690458a-3752-473c-8146-fe9dc8ccb17d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:08.123297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690458a-3752-473c-8146-fe9dc8ccb17d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:08.123512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4690458a-3752-473c-8146-fe9dc8ccb17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:08.132025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196dc4e-a86d-4b4d-9de7-231e76809464 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:08.143291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:08.172771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4690458a-3752-473c-8146-fe9dc8ccb17d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:08.174019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4690458a-3752-473c-8146-fe9dc8ccb17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:08.186114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:08.261524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:08.262715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:08.262996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:08.263276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] PATH_INFO: `/v3/users/67633cf5c142472d9e47291f6e11ad0d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:08.264701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67633cf5c142472d9e47291f6e11ad0d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:08.267171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:08.325879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 928/1852] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:08.332101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:08.334012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:08.335161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:08.348551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67633cf5c142472d9e47291f6e11ad0d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.388973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67633cf5c142472d9e47291f6e11ad0d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.425197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:08.426567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:08.426982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67633cf5c142472d9e47291f6e11ad0d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.427834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67633cf5c142472d9e47291f6e11ad0d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.453202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67633cf5c142472d9e47291f6e11ad0d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:08.465949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 929/1853] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:08.468904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:08.475230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 925/1854] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:08 2022] DELETE /identity/v3/users/67633cf5c142472d9e47291f6e11ad0d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:08.501451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a71e5-e024-406f-ba4f-c261ab90a518 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:08.645480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:08.647758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:08.649037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:08.653291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:08.655144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:08.655518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:08.661701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:08.718211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 930/1855] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:08.722053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:08.767040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a8d4-ec18-4a75-961b-29a43f48d6a7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:08.775786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adb45c-f6c7-4007-a32e-f7341ce406c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:08.776799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adb45c-f6c7-4007-a32e-f7341ce406c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:08.777637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adb45c-f6c7-4007-a32e-f7341ce406c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:08.777637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adb45c-f6c7-4007-a32e-f7341ce406c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:08.778654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adb45c-f6c7-4007-a32e-f7341ce406c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:08.784311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adb45c-f6c7-4007-a32e-f7341ce406c7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:08.956149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 926/1856] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:08 2022] POST /identity/v3/projects => generated 379 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:08.959644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:08.984821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a931d1d-3b3e-4e06-bb09-ba20cad1f4df admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.041983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757c076-3a1e-47d8-960a-b835a73552ae None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.043179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757c076-3a1e-47d8-960a-b835a73552ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.043834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2757c076-3a1e-47d8-960a-b835a73552ae None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.054361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 927/1857] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:46:09 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:46:09.086824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757c076-3a1e-47d8-960a-b835a73552ae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:09.091720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2757c076-3a1e-47d8-960a-b835a73552ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:09.149165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234f45d-7db2-40e6-9231-fc36f3e467df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.149930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234f45d-7db2-40e6-9231-fc36f3e467df admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.150396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234f45d-7db2-40e6-9231-fc36f3e467df admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.150773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234f45d-7db2-40e6-9231-fc36f3e467df admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.151688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234f45d-7db2-40e6-9231-fc36f3e467df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.155844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234f45d-7db2-40e6-9231-fc36f3e467df admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.223795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 931/1858] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:08 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:09.223795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.223795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.223795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.299869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:09.302273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:09.384764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2757c076-3a1e-47d8-960a-b835a73552ae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.400125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.401281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:09.406660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.407788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.408443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] PATH_INFO: `/v3/projects/f1f87e29338d46e6a37b63aebf01ecc0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.415022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1f87e29338d46e6a37b63aebf01ecc0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.423046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.435515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:09.529219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 932/1859] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:46:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:09.533827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:09.543539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1f87e29338d46e6a37b63aebf01ecc0', 'request_id': 'req-62b8ffe5-a487-40e0-abef-8071587fbdde'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:09.565821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2818-9f1d-496b-89d5-c0a25c325bc4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.566253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1f87e29338d46e6a37b63aebf01ecc0', 'request_id': 'req-62b8ffe5-a487-40e0-abef-8071587fbdde'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:09.580577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1f87e29338d46e6a37b63aebf01ecc0', 'request_id': 'req-62b8ffe5-a487-40e0-abef-8071587fbdde'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:09.599425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] The token cache is being invalidate because project f1f87e29338d46e6a37b63aebf01ecc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:09.599993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:09.602088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62b8ffe5-a487-40e0-abef-8071587fbdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:09.610213 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 928/1860] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:09 2022] DELETE /identity/v3/projects/f1f87e29338d46e6a37b63aebf01ecc0 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:09.614673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70022a46-5c84-4677-b9ef-8bfe05b467c0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.616745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70022a46-5c84-4677-b9ef-8bfe05b467c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.617757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70022a46-5c84-4677-b9ef-8bfe05b467c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.687848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.689107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.689777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.690426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-70022a46-5c84-4677-b9ef-8bfe05b467c0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:09.690690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.692069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.692460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-70022a46-5c84-4677-b9ef-8bfe05b467c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:09.697317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.713130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 929/1861] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:09.717379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.717661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.717934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.719511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 933/1862] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:09.724819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:09.728450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:09.751297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6823a-69f9-41cb-949f-309316f74f56 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.757126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:09.758706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.758929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:09.760058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.760505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.761270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.762556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.766380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.778384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 930/1863] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:09.779976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:09.795127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ae193-d7d5-4848-8bfd-9701b6a494ad None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.798170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 934/1864] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:09 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:09.813247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:09.821571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.822159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.822529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.823769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.826372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:09.826857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.831477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.832163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c1356-c407-4c36-9a3c-664ecf557dc1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.845387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be295c29-a055-4888-b373-139ebfca678d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.859723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 931/1865] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:09.861743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295c29-a055-4888-b373-139ebfca678d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.861743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295c29-a055-4888-b373-139ebfca678d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.861743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be295c29-a055-4888-b373-139ebfca678d admin admin] PATH_INFO: `/v3/projects/a178a35556c6480db83c7609529f0bbb/users/597f5de04c294746b48e06902c21c917/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.864007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:09.882614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbd84-90ef-46fd-ba06-2316f2e08476 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:09.890953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:09.892486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.893285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.894091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.895477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.899725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.907334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be295c29-a055-4888-b373-139ebfca678d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a178a35556c6480db83c7609529f0bbb, user_id=597f5de04c294746b48e06902c21c917, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:09.914192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be295c29-a055-4888-b373-139ebfca678d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:09.943270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 935/1866] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:09 2022] PUT /identity/v3/projects/a178a35556c6480db83c7609529f0bbb/users/597f5de04c294746b48e06902c21c917/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:09.955212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:09.955470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:09.955802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:09.969040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 932/1867] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:09 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:09.971372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.018933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] MFA Rules not processed for user `597f5de04c294746b48e06902c21c917`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:10.020668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:10.170210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4113a-12a4-4146-836a-5ce9d0052653 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.181560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.184156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.184785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.185246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.187339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.191092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.205030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 936/1868] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:10.205976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.207444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:10.232114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:10.243244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 933/1869] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:09 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:10.244777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.258057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8630f8d-1014-4994-b404-a35e0c2e7397 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.265919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.266563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.266989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.267266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.271183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.271183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.326981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 934/1870] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:10 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:10.330161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.332985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f52930-614f-4c31-86c7-fea71669bdad None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.356754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:10.442309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.443400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.443700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.444043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.449696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:10.488492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.495785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.496084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391a144-1e5c-4520-96bf-344e3b2b129a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.509543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.510728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.512133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.518402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.519509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.522372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.535770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 937/1871] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:10.539022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.543385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 935/1872] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:10.544702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.559630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4bf05-89dd-4699-9234-71c40bce15b9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.576536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65dc504-e2fe-44dc-8488-74e9fba25760 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.578333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.578962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.579281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.580604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.582347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.584355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.597308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.597308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.597308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.597308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.598584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.606183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.616205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 936/1873] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:10.638482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.652651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:10.691925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee98338-9595-4eb8-b4e8-4e0c27cac461 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.697168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 938/1874] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:10 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:10.698099 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.712395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.713085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.729283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.729283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] PATH_INFO: `/v3/projects/603f3324235e4e7aac2e5abdcc297066` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.734054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=603f3324235e4e7aac2e5abdcc297066)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.738851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.763600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5498f3-79c1-48c1-9035-438a2f042b53 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:10.806010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:10.806801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:10.807084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:10.807780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] PATH_INFO: `/v3/projects/d00b4ea7657d41e6a02531c3509619c9/users/30363c264f9b4ff5b5e17654bc3e9000/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:10.847220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00b4ea7657d41e6a02531c3509619c9, user_id=30363c264f9b4ff5b5e17654bc3e9000, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:10.850988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:10.890410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 939/1875] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:10 2022] PUT /identity/v3/projects/d00b4ea7657d41e6a02531c3509619c9/users/30363c264f9b4ff5b5e17654bc3e9000/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:10.898808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:10.984561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603f3324235e4e7aac2e5abdcc297066', 'request_id': 'req-f528bf24-43f6-4468-9acb-f52176716eb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.003651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603f3324235e4e7aac2e5abdcc297066', 'request_id': 'req-f528bf24-43f6-4468-9acb-f52176716eb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.014828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603f3324235e4e7aac2e5abdcc297066', 'request_id': 'req-f528bf24-43f6-4468-9acb-f52176716eb4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.047135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] The token cache is being invalidate because project 603f3324235e4e7aac2e5abdcc297066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:11.047768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.050077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f528bf24-43f6-4468-9acb-f52176716eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.055485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 937/1876] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:10 2022] DELETE /identity/v3/projects/603f3324235e4e7aac2e5abdcc297066 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:11.063185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.066184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.066777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.099934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:11.135621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1ada8-510c-47af-95ee-3c5eb7ba86a3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.139654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:11.145465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:11.312682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7a50e-b6ce-4660-96c0-20cefc507618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.313251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7a50e-b6ce-4660-96c0-20cefc507618 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.313637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7a50e-b6ce-4660-96c0-20cefc507618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.313800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7a50e-b6ce-4660-96c0-20cefc507618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.314402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7a50e-b6ce-4660-96c0-20cefc507618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.319934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde7a50e-b6ce-4660-96c0-20cefc507618 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.341276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 940/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:11.348136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375bc36-8b3a-4c35-9e97-0b2f3653f9c8 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.348423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375bc36-8b3a-4c35-9e97-0b2f3653f9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.349266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375bc36-8b3a-4c35-9e97-0b2f3653f9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.380417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 938/1878] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:11.382293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.386761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:11.410000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1375bc36-8b3a-4c35-9e97-0b2f3653f9c8 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:11.417271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc796c-3984-44d2-b324-ac3b4e7d0031 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.423817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1375bc36-8b3a-4c35-9e97-0b2f3653f9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:11.430978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.431862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.432161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.432511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] PATH_INFO: `/v3/projects/fa9bd982edd94fc2925d4bbacc02198b/users/103bf2713ec84bd58c818b5b121c98f5/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.460524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 941/1879] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:11.462170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.462694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.463020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.468677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9bd982edd94fc2925d4bbacc02198b, user_id=103bf2713ec84bd58c818b5b121c98f5, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.475298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.492630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:11.504237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:11.507151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:11.508466 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 939/1880] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:11 2022] PUT /identity/v3/projects/fa9bd982edd94fc2925d4bbacc02198b/users/103bf2713ec84bd58c818b5b121c98f5/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:11.509793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.514773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:11.628228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 942/1881] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:11.629095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.636163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc758aa-e601-43f7-97f4-1b1dad05b794 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.659383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.659566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.660143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.660495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.661364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f3312-f4f2-4ea3-9216-aaa01bbfe520 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.682140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.682774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.682884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.683067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.683347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.684018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.685353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.688901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.697756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 940/1882] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:11.698616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.705515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 943/1883] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:11.706537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.708484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:11.713727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738fd5b-ac5e-46c9-9449-d24cbb99d530 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.722244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.722686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.723169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.723318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d02b2d-0d82-4493-820c-cb08f833ff08 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.723551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.724325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:11.724679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.728443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.731153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.733126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.733126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.733126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] PATH_INFO: `/v3/users/032b078fb1b1451ea594f6ecaf2fd366` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.733953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032b078fb1b1451ea594f6ecaf2fd366)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.736297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.746224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 941/1884] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:11.747147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.761348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480b8d2-b968-4a53-b094-bcf567231907 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.769452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:11.770418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.770935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.771483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] PATH_INFO: `/v3/projects/fa9bd982edd94fc2925d4bbacc02198b/users/103bf2713ec84bd58c818b5b121c98f5/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.774189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa9bd982edd94fc2925d4bbacc02198b, user_id=103bf2713ec84bd58c818b5b121c98f5, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:11.777667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:11.792793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 942/1885] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:11 2022] PUT /identity/v3/projects/fa9bd982edd94fc2925d4bbacc02198b/users/103bf2713ec84bd58c818b5b121c98f5/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:11.793644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:11.815218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032b078fb1b1451ea594f6ecaf2fd366'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.835199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032b078fb1b1451ea594f6ecaf2fd366'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.856590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032b078fb1b1451ea594f6ecaf2fd366'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.860128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032b078fb1b1451ea594f6ecaf2fd366'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.879324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:11.884719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3684b5d7-ee40-433a-a396-aadb77e032a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032b078fb1b1451ea594f6ecaf2fd366'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:11.905061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 944/1886] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:11 2022] DELETE /identity/v3/users/032b078fb1b1451ea594f6ecaf2fd366 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:11.908538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c9bd2-4342-4050-96e3-a74e032c41fe admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:11.908644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c676e-85bd-4b2a-804e-82d159bd31c1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:11.908644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c676e-85bd-4b2a-804e-82d159bd31c1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:11.908644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c676e-85bd-4b2a-804e-82d159bd31c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:11.943927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5d2c676e-85bd-4b2a-804e-82d159bd31c1 None None] MFA Rules not processed for user `103bf2713ec84bd58c818b5b121c98f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:11.945617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5d2c676e-85bd-4b2a-804e-82d159bd31c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:12.014766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.016078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.016078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.016409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.017182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.019640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.069423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 945/1887] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:12.070562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.070892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.071215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.083285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 943/1888] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:11 2022] POST /identity/v3/projects => generated 383 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:12.092634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:12.096077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:12.102211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:12.103727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:12.127234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:12.227949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 946/1889] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:12.229192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:12.232401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:12.249151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683314d-3e26-4606-a7df-c325caf685d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:12.255670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.256475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.256819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.257162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.257891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.260320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.323588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 944/1890] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:12 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:12.323588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:12.329176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:12.332425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecb04b-6dc9-4832-ae2a-dee2479eadff None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:12.345943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.348168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.353134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.353664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.359735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:12.456417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca2b41-1551-45a2-b0b7-6c3c660685ae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:12.462831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:12.480846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.485249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.485601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.486227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.503090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.505152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.525995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 945/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:12.527809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:12.529753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.531351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.540683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27a59f-ebdb-4dab-9635-ea3ba3c53931 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:12.545802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.546419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.546687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.546953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.547628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.549541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.552919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 947/1892] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:12.579902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 946/1893] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:12.583950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:12.607799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55ea67-8d97-45e8-bfde-6ed83b97eeea service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:12.623045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390922af-6e7c-49ea-8a93-44adb326ae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.624157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390922af-6e7c-49ea-8a93-44adb326ae23 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.624735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390922af-6e7c-49ea-8a93-44adb326ae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.625324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390922af-6e7c-49ea-8a93-44adb326ae23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.625659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390922af-6e7c-49ea-8a93-44adb326ae23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.627953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390922af-6e7c-49ea-8a93-44adb326ae23 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.645560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 948/1894] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:12.647785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:12.661920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ed382-0145-4f6b-a8e1-52c4dedb9e93 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:12.669148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:12.669900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.670300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.670802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] PATH_INFO: `/v3/projects/cbc1d51c309440a8b2787c614a0e7d73/users/a5c51c76f45d45a4a3baa148bdb3b139/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.700332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc1d51c309440a8b2787c614a0e7d73, user_id=a5c51c76f45d45a4a3baa148bdb3b139, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:12.709926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:12.762884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 947/1895] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:12 2022] PUT /identity/v3/projects/cbc1d51c309440a8b2787c614a0e7d73/users/a5c51c76f45d45a4a3baa148bdb3b139/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:12.766160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:12.766491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:12.766809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:12.859784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] MFA Rules not processed for user `a5c51c76f45d45a4a3baa148bdb3b139`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:12.864278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.030606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 949/1896] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:13.080067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.187703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab663cc-ffed-448f-822e-c1e75724c1dd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:13.193812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43048c-070d-4f5a-83cd-2cc7e6a28f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:13.194597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43048c-070d-4f5a-83cd-2cc7e6a28f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.194946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43048c-070d-4f5a-83cd-2cc7e6a28f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.195425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43048c-070d-4f5a-83cd-2cc7e6a28f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.207084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43048c-070d-4f5a-83cd-2cc7e6a28f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:13.208801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43048c-070d-4f5a-83cd-2cc7e6a28f32 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:13.267675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 948/1897] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:13.322057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.346160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.346552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.346829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.374323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:13.411114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] MFA Rules not processed for user `30363c264f9b4ff5b5e17654bc3e9000`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:13.415395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.454210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af060e31-b9cc-4d7d-98ff-cd7bba639315 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:13.469691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:13.470646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.471103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.471596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] PATH_INFO: `/v3/users/a5c51c76f45d45a4a3baa148bdb3b139` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.473467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c51c76f45d45a4a3baa148bdb3b139)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:13.478617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:13.554471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 949/1898] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:13.561141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.566061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.578547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c51c76f45d45a4a3baa148bdb3b139'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.591333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c51c76f45d45a4a3baa148bdb3b139'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.593626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:13.605842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb44d0-dc1f-4a4f-b976-3af584d42221 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:13.614399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c51c76f45d45a4a3baa148bdb3b139'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.615396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c51c76f45d45a4a3baa148bdb3b139'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.623463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d567d-bd0f-4ce6-8f7c-840585c0b294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:13.625622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d567d-bd0f-4ce6-8f7c-840585c0b294 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.627024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d567d-bd0f-4ce6-8f7c-840585c0b294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.627873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d567d-bd0f-4ce6-8f7c-840585c0b294 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.629484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d567d-bd0f-4ce6-8f7c-840585c0b294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:13.634762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d567d-bd0f-4ce6-8f7c-840585c0b294 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:13.644604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c51c76f45d45a4a3baa148bdb3b139'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.653814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 950/1899] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:13 2022] DELETE /identity/v3/users/a5c51c76f45d45a4a3baa148bdb3b139 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:13.658327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.659927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.716104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 950/1900] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:13 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:13.718236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.718747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.719456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.761135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:13.762611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.791052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc294e5-7fb8-4127-8278-8394e4f4202e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:13.844962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 951/1901] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:13.845698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.847028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.862189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58beb29a-14b2-496a-8557-734a3cf9d023 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:13.870480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:13.871423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.871850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.872104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.872979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:13.875327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:13.884253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:13.884908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:13.885182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:13.885451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] PATH_INFO: `/v3/projects/0f6964429ff54bb0875a37c8d323562d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:13.886590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f6964429ff54bb0875a37c8d323562d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:13.888673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:13.905999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 952/1902] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:13 2022] POST /identity/v3/users => generated 292 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:13.907122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.910649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:13.960300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f6964429ff54bb0875a37c8d323562d', 'request_id': 'req-5f0cebeb-4c54-42cf-8b7e-d43886882a80'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.966739 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f6964429ff54bb0875a37c8d323562d', 'request_id': 'req-5f0cebeb-4c54-42cf-8b7e-d43886882a80'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.973652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f6964429ff54bb0875a37c8d323562d', 'request_id': 'req-5f0cebeb-4c54-42cf-8b7e-d43886882a80'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.984241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398cb5e-d9ec-48cb-b9c9-a55de1860f2d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:13.988972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] The token cache is being invalidate because project 0f6964429ff54bb0875a37c8d323562d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:13.989264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.991569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:13.993677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 951/1903] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:13 2022] DELETE /identity/v3/projects/0f6964429ff54bb0875a37c8d323562d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:13.994888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:13.996455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:13.998983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.019567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:14.056824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.057769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.058205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.058655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.061199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.092231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cebeb-4c54-42cf-8b7e-d43886882a80 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.096873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.097477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.097753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.098017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.099352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.100823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.111271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 952/1904] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:14.112151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:14.113822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.127634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfb35f-a81d-47f5-a736-aec5d3334797 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.134291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.135482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.135998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.136647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.136934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.137207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] PATH_INFO: `/v3/users/198b2a477716424ebcd479a2e5606dfd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.138358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198b2a477716424ebcd479a2e5606dfd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.140606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.146440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 953/1905] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:14.147106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:14.156966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbce44-6489-468a-8851-467092241ccd service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.163079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.163917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.164689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.164689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] PATH_INFO: `/v3/projects/9ca129e74fe049d5a345febe9321d2e8/users/5be6d6edaabf476ca5189b11f44f275e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.185910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca129e74fe049d5a345febe9321d2e8, user_id=5be6d6edaabf476ca5189b11f44f275e, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.189539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.200971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198b2a477716424ebcd479a2e5606dfd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:14.208132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 954/1906] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:14 2022] PUT /identity/v3/projects/9ca129e74fe049d5a345febe9321d2e8/users/5be6d6edaabf476ca5189b11f44f275e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:14.210000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:14.214009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198b2a477716424ebcd479a2e5606dfd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:14.228578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198b2a477716424ebcd479a2e5606dfd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:14.230263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198b2a477716424ebcd479a2e5606dfd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:14.266557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198b2a477716424ebcd479a2e5606dfd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:14.288462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 953/1907] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:14 2022] DELETE /identity/v3/users/198b2a477716424ebcd479a2e5606dfd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:14.322687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3cba81-5793-4b78-a583-56e9768793c9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.331873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:14.336205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.388808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:14.407961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.408914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.408914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.409102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.409740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.411588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.427388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 955/1908] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:14.432224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:14.434913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.442315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ff10c-0b0b-4d6d-a267-45fd3283a758 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.453804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.454444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.455008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.455115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.456574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a2c71-45ae-4fc5-9496-8e327cd2e8da admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.458901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.465622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.466793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.467268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.468602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] PATH_INFO: `/v3/projects/9ca129e74fe049d5a345febe9321d2e8/users/5be6d6edaabf476ca5189b11f44f275e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.470990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca129e74fe049d5a345febe9321d2e8, user_id=5be6d6edaabf476ca5189b11f44f275e, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.474303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.501105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:14.539710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 956/1909] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:14 2022] PUT /identity/v3/projects/9ca129e74fe049d5a345febe9321d2e8/users/5be6d6edaabf476ca5189b11f44f275e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:14.582974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.585398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be67d8-e1e1-411a-bbbd-85cfe8bdd581 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:14.623615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 954/1910] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:14.831793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:14.835248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:14.975349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bb3da-eb29-40fe-bea7-2d2ee0a52507 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:14.981415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:14.982349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:14.982685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:14.982936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] PATH_INFO: `/v3/projects/cbc1d51c309440a8b2787c614a0e7d73` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:14.984336 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc1d51c309440a8b2787c614a0e7d73)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:14.986438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.051235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc1d51c309440a8b2787c614a0e7d73', 'request_id': 'req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.058908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc1d51c309440a8b2787c614a0e7d73', 'request_id': 'req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.072080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc1d51c309440a8b2787c614a0e7d73', 'request_id': 'req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.090869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] The token cache is being invalidate because project cbc1d51c309440a8b2787c614a0e7d73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:15.091703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.092796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.095661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 957/1911] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:14 2022] DELETE /identity/v3/projects/cbc1d51c309440a8b2787c614a0e7d73 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:15.103422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982083-e628-4e87-becd-6ada32a75bdb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.103906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982083-e628-4e87-becd-6ada32a75bdb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.104341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982083-e628-4e87-becd-6ada32a75bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.111920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.114653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:15.141699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-db982083-e628-4e87-becd-6ada32a75bdb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:15.146066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-db982083-e628-4e87-becd-6ada32a75bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:15.265783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbd7e-5b91-4bf0-8b22-da9456b1ae20 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.274138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.274802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.275193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.275636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] PATH_INFO: `/v3/projects/69a7cede1c214895a4e6e0eb47eb122b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.278218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a7cede1c214895a4e6e0eb47eb122b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.280826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.282848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:15.310214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 955/1912] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:15.318028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.318308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.318616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.347899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:15.349250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:15.354892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a7cede1c214895a4e6e0eb47eb122b', 'request_id': 'req-d5cb7b5b-e232-4ed7-87e6-937cfd444224'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.364707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a7cede1c214895a4e6e0eb47eb122b', 'request_id': 'req-d5cb7b5b-e232-4ed7-87e6-937cfd444224'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.372359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a7cede1c214895a4e6e0eb47eb122b', 'request_id': 'req-d5cb7b5b-e232-4ed7-87e6-937cfd444224'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.392227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] The token cache is being invalidate because project 69a7cede1c214895a4e6e0eb47eb122b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:15.392935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.395254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cb7b5b-e232-4ed7-87e6-937cfd444224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:15.398456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 958/1913] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:15 2022] DELETE /identity/v3/projects/69a7cede1c214895a4e6e0eb47eb122b => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:15.405030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005ad6-0724-4270-afbe-71906e789fc8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.406092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005ad6-0724-4270-afbe-71906e789fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.406634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6005ad6-0724-4270-afbe-71906e789fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.426808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 956/1914] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:15.430128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6005ad6-0724-4270-afbe-71906e789fc8 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:15.431607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6005ad6-0724-4270-afbe-71906e789fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:15.431781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.440356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 959/1915] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:15.444038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e6500-263e-40c9-b50d-5ac8d6d18f80 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.448705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.448992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.449371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.449554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.450180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.450466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.450720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.451371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:15.451664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.458702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.473881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 957/1916] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:46:15.475910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:15.477156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:15.479386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.490695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ff365-341b-4e8f-953f-5ba11603c346 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.492606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 960/1917] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:15.498159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.499078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.499972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.500334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.500808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.501634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.504346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.514663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd1d99-49fa-47fc-866a-48ce4cc8b4dc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.520516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.521257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.521580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.521965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.522730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:15.523101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.525862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.545406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 961/1918] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:15.546337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.549326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 958/1919] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:15.554239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.564548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db9672-d8e6-438f-b088-9b72e32814ab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.640835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:15.680223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.681375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.681774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.682211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.683105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.685194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.693387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a16a-0b8f-43dd-a4b1-c389cdb4a8a7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.699390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb29a2-fc85-45c9-adc5-20c45d9407e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.700163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb29a2-fc85-45c9-adc5-20c45d9407e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.700444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb29a2-fc85-45c9-adc5-20c45d9407e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.700716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb29a2-fc85-45c9-adc5-20c45d9407e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.701412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb29a2-fc85-45c9-adc5-20c45d9407e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.704625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb29a2-fc85-45c9-adc5-20c45d9407e5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.747560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 959/1920] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:15.750900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.751183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.751446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 962/1921] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/projects => generated 365 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:15.752386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.753012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.839895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:15.858514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea09b7-0a78-405d-a1d7-7fcd0286df77 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.865392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] MFA Rules not processed for user `5be6d6edaabf476ca5189b11f44f275e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:15.866210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.866950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.867349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.867652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.868475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:15.868615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.870777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.884754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 963/1922] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:15.885634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:15.908074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ef08c-b011-428f-91ba-7bd3cd4719d2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:15.927383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:15.927383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:15.927383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:15.927383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:15.927710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:15.927710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:15.976455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 964/1923] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:15.984637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.004124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f850270-edf8-4fe1-9ee4-59202419f533 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.018024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.019288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.019844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.020493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.022962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.028225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.042913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 960/1924] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:16.055285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 965/1925] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:15 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:16.055726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.057552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.071933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127991d0-5c3e-4234-9b64-056f378a08e8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.078420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d24c-7db1-4423-a798-c5cb72ab813d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.084094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0a999-5c1b-443e-921b-ab16b0fa5ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.084929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a999-5c1b-443e-921b-ab16b0fa5ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.085377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.085924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a999-5c1b-443e-921b-ab16b0fa5ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.086103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.086335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0a999-5c1b-443e-921b-ab16b0fa5ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.086435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.086619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] PATH_INFO: `/v3/projects/82ad5a27d771442aa29b4a0030e275e3/users/a9e7bdbd9582440093d88457c5bd053f/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.087189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0a999-5c1b-443e-921b-ab16b0fa5ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.089051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c0a999-5c1b-443e-921b-ab16b0fa5ba6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.105653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ad5a27d771442aa29b4a0030e275e3, user_id=a9e7bdbd9582440093d88457c5bd053f, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.110396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 961/1926] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:16.110396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4fc2-8747-45d2-9b1c-e89f3d80fcad None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.110760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.110834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4fc2-8747-45d2-9b1c-e89f3d80fcad None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.110834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f4fc2-8747-45d2-9b1c-e89f3d80fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.128282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 966/1927] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:16 2022] PUT /identity/v3/projects/82ad5a27d771442aa29b4a0030e275e3/users/a9e7bdbd9582440093d88457c5bd053f/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:16.129203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.130963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5b1f4fc2-8747-45d2-9b1c-e89f3d80fcad None None] MFA Rules not processed for user `30363c264f9b4ff5b5e17654bc3e9000`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:16.132475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5b1f4fc2-8747-45d2-9b1c-e89f3d80fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:16.238272 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceddd1-a648-40fc-8e1a-21deb90a6a6c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.247052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 962/1928] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:16.249384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.249713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.250088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.250681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.258556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.258556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.258556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.258556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.258556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.271155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 967/1929] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:16 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:16.271639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.275413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:16.324749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] MFA Rules not processed for user `a9e7bdbd9582440093d88457c5bd053f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:16.326016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:16.386757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:16.418333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680da3c5-6eb8-4aad-81b3-ef29c9e2335e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.434877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.437552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.438003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.438280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.445599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:16.459024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 963/1930] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:16.459024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.493760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.497661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.498288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6191a-96cd-4fef-bbca-c42610fb3d63 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.511430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a436e3-a9b8-4225-954d-ffb635948300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.512811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a436e3-a9b8-4225-954d-ffb635948300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.513105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a436e3-a9b8-4225-954d-ffb635948300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.513460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a436e3-a9b8-4225-954d-ffb635948300 admin admin] PATH_INFO: `/v3/projects/ab05b46b3f81484b80daf796469d4db0/users/6598ccba35b346ea933ec1ceb36db48f/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.520429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 968/1931] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:16.523026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.529249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:16.552730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a436e3-a9b8-4225-954d-ffb635948300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab05b46b3f81484b80daf796469d4db0, user_id=6598ccba35b346ea933ec1ceb36db48f, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.558329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:16.558508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a436e3-a9b8-4225-954d-ffb635948300 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.580757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 964/1932] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:16 2022] PUT /identity/v3/projects/ab05b46b3f81484b80daf796469d4db0/users/6598ccba35b346ea933ec1ceb36db48f/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:16.593050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.593458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.593824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.638186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a8cd2-d1bc-4b3b-a416-69489e63457f service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.639866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] MFA Rules not processed for user `6598ccba35b346ea933ec1ceb36db48f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:16.641103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:16.712149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.713701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.714881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.715159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.726414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 965/1933] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:16.728962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.826038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.827943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.840131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 969/1934] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:16.848563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a574fc6-7846-4df5-982f-9360ef6d4383 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.869208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.870080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.870989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.871397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.871802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.872547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.874581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.885449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21032486-582a-4a5c-a927-27bc9d6661e2 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:16.892468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:16.893113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:16.893388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:16.893659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] PATH_INFO: `/v3/users/a9e7bdbd9582440093d88457c5bd053f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:16.897726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9e7bdbd9582440093d88457c5bd053f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:16.900406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:16.916598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 966/1935] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:16 2022] POST /identity/v3/projects => generated 365 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:16.920230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:16.963450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:16.981575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9e7bdbd9582440093d88457c5bd053f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:16.991449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9e7bdbd9582440093d88457c5bd053f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:17.008142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9e7bdbd9582440093d88457c5bd053f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:17.008872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9e7bdbd9582440093d88457c5bd053f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:17.012500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a10e3-7389-445d-9c24-4b784dfaae55 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:17.015560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:17.017647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4dddafd3-4413-49a4-aa97-cf9a17909a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9e7bdbd9582440093d88457c5bd053f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:17.019472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:17.020057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.020329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.020740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.021219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:17.023908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:17.027504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 970/1936] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:16 2022] DELETE /identity/v3/users/a9e7bdbd9582440093d88457c5bd053f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:17.032059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.032462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.032822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.051199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 967/1937] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:16 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:17.055353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:17.057311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:17.060503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:17.062257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:17.072860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fe4d9-b4ba-47dc-9e37-4a51a3122734 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:17.159567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:17.160267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.160543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.160812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.161456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:17.163105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:17.185972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 971/1938] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:17.196775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:17.198689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:17.200115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 968/1939] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:17.204731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:17.225233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa8e22-143a-4be5-854c-f570b8f21292 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:17.238931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:17.239965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.240255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.240534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] PATH_INFO: `/v3/projects/41f9f4e01e0446759c3d6b0d0b4fead6/users/5176c22c411548c38fb404bbdb22aa38/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.285418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f9f4e01e0446759c3d6b0d0b4fead6, user_id=5176c22c411548c38fb404bbdb22aa38, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:17.288945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:17.309177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 969/1940] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:17 2022] PUT /identity/v3/projects/41f9f4e01e0446759c3d6b0d0b4fead6/users/5176c22c411548c38fb404bbdb22aa38/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:17.314481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c050e-5922-43ab-a443-fc5af2ac1223 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:17.314973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:17.408170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fa694-929e-4151-803b-922151f2937c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:17.409146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa694-929e-4151-803b-922151f2937c service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.409434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa694-929e-4151-803b-922151f2937c service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.409707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fa694-929e-4151-803b-922151f2937c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.487603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6acb2f-13ae-4d46-942f-5917e8023328 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:17.519693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:17.519693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.519693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.519693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.520714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:17.520714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:17.546192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 970/1941] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:17.550444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:17.568580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fa694-929e-4151-803b-922151f2937c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:17.571212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fa694-929e-4151-803b-922151f2937c service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:17.573720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907ce1c-4e17-4711-80a8-9d7abfcc7470 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:17.581283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:17.582219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.582892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.583363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] PATH_INFO: `/v3/projects/41f9f4e01e0446759c3d6b0d0b4fead6/users/5176c22c411548c38fb404bbdb22aa38/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.587506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f9f4e01e0446759c3d6b0d0b4fead6, user_id=5176c22c411548c38fb404bbdb22aa38, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:17.590953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:17.593079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 972/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:17.604568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 971/1943] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:17 2022] PUT /identity/v3/projects/41f9f4e01e0446759c3d6b0d0b4fead6/users/5176c22c411548c38fb404bbdb22aa38/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:17.615030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:17.616722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:17.617166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:17.626548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:17.715684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] MFA Rules not processed for user `5176c22c411548c38fb404bbdb22aa38`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:17.719519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:17.921248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 973/1944] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:17.935225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:17.937163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:17.955422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:18.029310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ff8b2-ea77-4490-a114-47a9c4ab6b1e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:18.036262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:18.036940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.038065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.038394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.050902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:18.052483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:18.065311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 972/1945] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:18.231703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:18.247098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bfc7e-561e-47c3-852b-a343ab636121 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:18.257790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:18.258438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.264236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.264614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.268496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:18.274300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:18.275671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:18.385821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d32c1d-a032-4be4-a8ee-08a099b7229a service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:18.394313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:18.395010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.395326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.395529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] PATH_INFO: `/v3/projects/82ad5a27d771442aa29b4a0030e275e3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.397600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ad5a27d771442aa29b4a0030e275e3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:18.399788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:18.415375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:18.417005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd31fd-bbe3-4467-b4cf-40c1c997e217 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:18.427869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 974/1946] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:18.468045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ad5a27d771442aa29b4a0030e275e3', 'request_id': 'req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:18.475713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ad5a27d771442aa29b4a0030e275e3', 'request_id': 'req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:18.483707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ad5a27d771442aa29b4a0030e275e3', 'request_id': 'req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:18.499634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] The token cache is being invalidate because project 82ad5a27d771442aa29b4a0030e275e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:18.500235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:18.500857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-02e8727e-f35d-4c9e-a0d7-69624f3c92f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:18.502566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 973/1947] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:18 2022] DELETE /identity/v3/projects/82ad5a27d771442aa29b4a0030e275e3 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:18.508088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.508363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.508636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.539262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:18.541169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:18.674855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 975/1948] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:18.685365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.685846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.686222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.746899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:18.750504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:18.766921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 974/1949] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:18.772367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:18.784405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188a8d5-b37d-44d9-a445-e65e4861d884 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:18.806886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:18.807988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.808273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.808547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.809278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:18.809588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:18.812008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:18.831365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 976/1950] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:18.836400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:18.851078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3b165-792f-4808-8863-84eb8d854119 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:18.857208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:18.858180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:18.859237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:18.859556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:18.860507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:18.863488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:18.921315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 975/1951] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:18 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:18.929578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:18.941953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84f54d-40be-4029-a045-41e72cf3ab70 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:18.953844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:18.955622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:18.956199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:18.971030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:19.054657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.054657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.054996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.055378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.057507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.060928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.084361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f2d8a-6296-4d0f-a05d-36b6dd82bca9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:19.091107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.091870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.092151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.092427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] PATH_INFO: `/v3/users/597f5de04c294746b48e06902c21c917` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.093779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597f5de04c294746b48e06902c21c917)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.095780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.124570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 977/1952] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:18 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:19.125570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:19.161099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cec16c-fc8b-4512-9f00-9c17897c94cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:19.174537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.175159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.175435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.176823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.177554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.182698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.193887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597f5de04c294746b48e06902c21c917'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:19.213660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597f5de04c294746b48e06902c21c917'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:19.214841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 978/1953] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:19.216348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:19.236715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c64ce8-f67c-4c96-8091-23a47da6e8a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:19.241382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597f5de04c294746b48e06902c21c917'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:19.243397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597f5de04c294746b48e06902c21c917'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:19.255320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.256191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.256436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.256705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.258864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.266679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.280411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9b01f394-516b-45aa-ba74-df4eec9b9725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597f5de04c294746b48e06902c21c917'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:19.291968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 979/1954] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:19.297849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 976/1955] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:18 2022] DELETE /identity/v3/users/597f5de04c294746b48e06902c21c917 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:19.298039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:19.298039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:19.299579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.299909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.300186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.319693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3175b0e-2284-489b-b182-38748d19edb9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:19.327585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:19.329632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:19.426618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 977/1956] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:19.434114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c54b8-b731-41bb-a93e-8ed6f91d85f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.435430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c54b8-b731-41bb-a93e-8ed6f91d85f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.435706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c54b8-b731-41bb-a93e-8ed6f91d85f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.436011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c54b8-b731-41bb-a93e-8ed6f91d85f4 admin admin] PATH_INFO: `/v3/projects/67e855fc220b42a5a5e4d3ed7700b59c/users/704b2d3d6c8243e0b71f18362248ec63/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.448256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:19.449864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:19.474784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c54b8-b731-41bb-a93e-8ed6f91d85f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e855fc220b42a5a5e4d3ed7700b59c, user_id=704b2d3d6c8243e0b71f18362248ec63, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.480599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c54b8-b731-41bb-a93e-8ed6f91d85f4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.501353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 980/1957] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:19 2022] PUT /identity/v3/projects/67e855fc220b42a5a5e4d3ed7700b59c/users/704b2d3d6c8243e0b71f18362248ec63/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:19.517444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.517741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.518861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.557396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:19.610535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] MFA Rules not processed for user `704b2d3d6c8243e0b71f18362248ec63`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:19.612006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:19.631338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87202645-726d-4cc8-8619-3ba6a2ab13d6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:19.710902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:19.731706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.753684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 981/1958] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:19.755595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.760854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.766483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.823159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:19.825978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:19.872963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.874505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.890003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 978/1959] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:19.943134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088ef37-f981-4d11-a5af-063aedada44c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:19.951913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:19.952732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:19.952732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:19.953007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:19.963996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:19.965689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:19.980570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 982/1960] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:20.553936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:20.555893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:20.585398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810276c8-5887-4880-940c-f462cb527d52 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:20.627078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:20.627784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:20.628067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:20.628341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] PATH_INFO: `/v3/projects/a178a35556c6480db83c7609529f0bbb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:20.636687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a178a35556c6480db83c7609529f0bbb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:20.641949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:20.790357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a178a35556c6480db83c7609529f0bbb', 'request_id': 'req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:20.803559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a178a35556c6480db83c7609529f0bbb', 'request_id': 'req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:20.816768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a178a35556c6480db83c7609529f0bbb', 'request_id': 'req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:20.861031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] The token cache is being invalidate because project a178a35556c6480db83c7609529f0bbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:20.861328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:20.862385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:20.868507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 979/1961] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:20 2022] DELETE /identity/v3/projects/a178a35556c6480db83c7609529f0bbb => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:20.870279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:20.890447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc8a09-3e3c-461e-ad47-50cb3e1f9f1e service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:20.992942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:20.993877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:20.994152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:20.994548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] PATH_INFO: `/v3/users/103bf2713ec84bd58c818b5b121c98f5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:20.995809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] RBAC: Authorizing `identity:delete_user(user_id=103bf2713ec84bd58c818b5b121c98f5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:20.997862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:21.046483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '103bf2713ec84bd58c818b5b121c98f5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.055471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '103bf2713ec84bd58c818b5b121c98f5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.078258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '103bf2713ec84bd58c818b5b121c98f5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.079232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '103bf2713ec84bd58c818b5b121c98f5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.089788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c304450-0183-4306-8962-fa86d5c7951d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '103bf2713ec84bd58c818b5b121c98f5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.095788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 983/1962] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:20 2022] DELETE /identity/v3/users/103bf2713ec84bd58c818b5b121c98f5 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:21.411474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:21.414049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:21.435217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c11d8-d74a-4b8c-9003-9814e88b27c4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:21.478053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:21.524096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:21.524868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:21.525143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:21.525414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] PATH_INFO: `/v3/projects/fa9bd982edd94fc2925d4bbacc02198b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:21.526599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa9bd982edd94fc2925d4bbacc02198b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:21.528813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:21.588791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa9bd982edd94fc2925d4bbacc02198b', 'request_id': 'req-15e76067-3241-42db-9ed9-0536c44e2ac1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.594710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa9bd982edd94fc2925d4bbacc02198b', 'request_id': 'req-15e76067-3241-42db-9ed9-0536c44e2ac1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.604251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa9bd982edd94fc2925d4bbacc02198b', 'request_id': 'req-15e76067-3241-42db-9ed9-0536c44e2ac1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.625286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] The token cache is being invalidate because project fa9bd982edd94fc2925d4bbacc02198b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:21.625681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.626616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-15e76067-3241-42db-9ed9-0536c44e2ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:21.628406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 980/1963] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:21 2022] DELETE /identity/v3/projects/fa9bd982edd94fc2925d4bbacc02198b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:21.634747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:21.634747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:21.634747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:21.659481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:21.664607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:21.676022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:21.736799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 984/1964] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:21.744990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:21.745285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:21.745567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:21.767799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:21.769033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:21.786450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 981/1965] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:21.793406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:21.804807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689b7e5-34ce-4635-b51c-ab36b6c5603e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:21.810284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:21.810923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:21.811205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:21.811493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:21.812226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:21.812531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:21.814332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:21.838476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 985/1966] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:21.839104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:21.848749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3739cf6-e937-4c84-880b-f8c1d3e4161a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:21.857480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:21.858734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:21.858734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:21.858956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:21.859611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:21.861706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:21.910323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 982/1967] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:21 2022] POST /identity/v3/projects => generated 379 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:21.915121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:21.929974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921c57e-09c3-4a57-9f5c-61c4f95da8cc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:22.083982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:22.085654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.086329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.086688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.087621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:22.090070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:22.125717 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 986/1968] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:21 2022] POST /identity/v3/users => generated 290 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:22.129865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:22.145028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547fe92-c4a8-4a19-a15e-5fa0ffcc2f4e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:22.172475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:22.172475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.172475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.172475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.172897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:22.172897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:22.186396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 983/1969] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:22.191358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:22.221423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed01cc-7c33-47fb-819e-d6b620edf5bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:22.227558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3e5ef-2ebe-4e49-a33f-1cbc64f6fb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:22.228601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3e5ef-2ebe-4e49-a33f-1cbc64f6fb7d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.228964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3e5ef-2ebe-4e49-a33f-1cbc64f6fb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.229188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3e5ef-2ebe-4e49-a33f-1cbc64f6fb7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.229905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3e5ef-2ebe-4e49-a33f-1cbc64f6fb7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:22.232123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3e5ef-2ebe-4e49-a33f-1cbc64f6fb7d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:22.248042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 987/1970] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:22.252783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:22.268344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80488d60-082d-4ffb-ab8c-bb0df648b398 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:22.275466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:22.276240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.280725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.280725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] PATH_INFO: `/v3/projects/cba313d1e2b04c72be4909db2d95e16e/users/e13b080af6244fe2a48d5af66a803d45/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.313171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba313d1e2b04c72be4909db2d95e16e, user_id=e13b080af6244fe2a48d5af66a803d45, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:22.316181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:22.350581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 984/1971] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:22 2022] PUT /identity/v3/projects/cba313d1e2b04c72be4909db2d95e16e/users/e13b080af6244fe2a48d5af66a803d45/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:22.352039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.352556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.353922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.419882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] MFA Rules not processed for user `e13b080af6244fe2a48d5af66a803d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:22.421124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:22.567981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 988/1972] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:22.569384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:22.636624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:22.690449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e416c-34b7-4221-9666-aff50f6dd025 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:22.705064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03928673-3072-4762-9736-808237effc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:22.705699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928673-3072-4762-9736-808237effc54 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.705973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928673-3072-4762-9736-808237effc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.706242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03928673-3072-4762-9736-808237effc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.706876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928673-3072-4762-9736-808237effc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:22.725648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03928673-3072-4762-9736-808237effc54 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:22.831558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 985/1973] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:22 2022] POST /identity/v3/projects => generated 381 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:22.833173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:22.853855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de897f-1340-4d7a-acee-36cc4e649999 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:22.872948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:22.987346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:22.989224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:22.990051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:22.990573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:22.993101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:22.997122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:23.058356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 989/1974] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:22 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:23.069263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03928673-3072-4762-9736-808237effc54 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:23.087018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03928673-3072-4762-9736-808237effc54 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:23.094847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:23.095501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:23.096838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:23.097226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:23.097901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:23.099525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:23.118677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 986/1975] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:23.122819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:23.160280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c421fc-76ff-46d4-a541-a5316ff842cc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:23.196151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:23.196967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:23.197424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:23.197892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] PATH_INFO: `/v3/projects/8e1b780581554100bfb6fe1116e6f3eb/users/599dd2c04c5a4b169213c44c6a223e42/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:23.217726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1b780581554100bfb6fe1116e6f3eb, user_id=599dd2c04c5a4b169213c44c6a223e42, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:23.221108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:23.247790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 990/1976] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:23 2022] PUT /identity/v3/projects/8e1b780581554100bfb6fe1116e6f3eb/users/599dd2c04c5a4b169213c44c6a223e42/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:23.250470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:23.277408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d7ecf-61e4-4e5e-b1fb-1b37f87b415a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:23.398502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0149d72-ad9c-4338-9715-25a1926b4b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:23.399269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0149d72-ad9c-4338-9715-25a1926b4b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:23.399554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0149d72-ad9c-4338-9715-25a1926b4b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:23.399849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0149d72-ad9c-4338-9715-25a1926b4b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:23.400496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0149d72-ad9c-4338-9715-25a1926b4b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:23.402177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0149d72-ad9c-4338-9715-25a1926b4b25 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:23.427403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 987/1977] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:23.431618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:23.452433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc3416-ca8e-429c-b927-dd96b1d9924a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:23.469038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:23.470064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:23.470574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:23.471040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] PATH_INFO: `/v3/projects/8e1b780581554100bfb6fe1116e6f3eb/users/599dd2c04c5a4b169213c44c6a223e42/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:23.477302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1b780581554100bfb6fe1116e6f3eb, user_id=599dd2c04c5a4b169213c44c6a223e42, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:23.480994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:23.494969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 991/1978] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:23 2022] PUT /identity/v3/projects/8e1b780581554100bfb6fe1116e6f3eb/users/599dd2c04c5a4b169213c44c6a223e42/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:23.505598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:23.505957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:23.506807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:23.540590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] MFA Rules not processed for user `599dd2c04c5a4b169213c44c6a223e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:23.541943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:23.660280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 988/1979] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:23.682456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:23.684264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:23.760500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d938c08e-8549-4ea2-b1fa-5e3c8b4281ff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:23.765183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:23.765924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:23.767635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:23.767635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:23.781563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:23.783606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:23.797451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 992/1980] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:24.411558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:24.422872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136b79d-9f0c-45c1-af3d-28de91552735 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:24.427855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599dd2c04c5a4b169213c44c6a223e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e1b780581554100bfb6fe1116e6f3eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:24.428525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:24.429508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:24.429799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:24.430461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:24.430766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:24.432652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:24.451111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 989/1981] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:46:24.455012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:24.470209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77b4a2-3c79-444b-96ee-8f812a20d325 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:24.476106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:24.478439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:24.486143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599dd2c04c5a4b169213c44c6a223e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e1b780581554100bfb6fe1116e6f3eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:24.486773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:24.487245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:24.487666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:24.489248 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:24.492518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:24.534333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:24.594582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 993/1982] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:24 2022] POST /identity/v3/projects => generated 354 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:24.598614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:24.604628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59c1ac-0c91-4240-9381-df0e7e54c2f9 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:24.655322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:24.684958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f2d08-3fc4-4eec-b654-d11154e9dc02 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:24.692433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599dd2c04c5a4b169213c44c6a223e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e1b780581554100bfb6fe1116e6f3eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:24.693241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:24.693388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:24.694626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:24.694626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:24.695036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:24.695256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:24.695420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:24.696168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:24.697216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] PATH_INFO: `/v3/users/704b2d3d6c8243e0b71f18362248ec63` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:24.697514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:24.701277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=704b2d3d6c8243e0b71f18362248ec63)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:24.703884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:24.715348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 994/1983] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:46:24.777879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:24.793682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '704b2d3d6c8243e0b71f18362248ec63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.795237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e4e9d-6b62-4011-8e57-706681d17831 tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:24.804065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '599dd2c04c5a4b169213c44c6a223e42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e1b780581554100bfb6fe1116e6f3eb', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:24.805119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:24.805585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:24.806583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] PATH_INFO: `/v3/projects/010cd46ba16c4d888f26a368e91c57db` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:24.808143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorizing `identity:delete_project(project_id=010cd46ba16c4d888f26a368e91c57db)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:24.809511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '704b2d3d6c8243e0b71f18362248ec63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.810770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:24.824128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '704b2d3d6c8243e0b71f18362248ec63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.825591 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '704b2d3d6c8243e0b71f18362248ec63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.845682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-58292772-aeac-4dbb-a867-28766645ec1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '704b2d3d6c8243e0b71f18362248ec63'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.899954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 990/1984] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:24 2022] DELETE /identity/v3/users/704b2d3d6c8243e0b71f18362248ec63 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:24.901458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:24.902493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:24.902493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:24.920813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010cd46ba16c4d888f26a368e91c57db', 'request_id': 'req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.938097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010cd46ba16c4d888f26a368e91c57db', 'request_id': 'req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.956964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:24.958232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:24.960109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010cd46ba16c4d888f26a368e91c57db', 'request_id': 'req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.989249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] The token cache is being invalidate because project 010cd46ba16c4d888f26a368e91c57db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:24.989568 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.991507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:24.999613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 995/1985] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:24 2022] DELETE /identity/v3/projects/010cd46ba16c4d888f26a368e91c57db => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:25.065074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:25.100251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 991/1986] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:25.110940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:25.112399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:25.215505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c87ef-6a5b-47ed-9b01-117112cf5f2e tempest-VolumeQuotaClassesTest-1156952191 tempest-VolumeQuotaClassesTest-1156952191-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:25.340110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1bf85-4533-4398-a46d-8bdffea27bac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:25.341066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1bf85-4533-4398-a46d-8bdffea27bac service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:25.341617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1bf85-4533-4398-a46d-8bdffea27bac service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:25.342103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1bf85-4533-4398-a46d-8bdffea27bac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:25.388781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1bf85-4533-4398-a46d-8bdffea27bac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:25.396053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1bf85-4533-4398-a46d-8bdffea27bac service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:25.431915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 996/1987] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:25.652133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:25.656595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:25.674892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bb565-5643-4bc5-b084-8ff294a1e55b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:25.682986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:25.684712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:25.685240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:25.685944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] PATH_INFO: `/v3/users/e13b080af6244fe2a48d5af66a803d45` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:25.687898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13b080af6244fe2a48d5af66a803d45)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:25.690638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:25.807934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13b080af6244fe2a48d5af66a803d45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:25.818518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13b080af6244fe2a48d5af66a803d45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:25.827543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13b080af6244fe2a48d5af66a803d45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:25.828673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13b080af6244fe2a48d5af66a803d45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:25.852240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13b080af6244fe2a48d5af66a803d45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:25.863923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 992/1988] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:25 2022] DELETE /identity/v3/users/e13b080af6244fe2a48d5af66a803d45 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:25.869363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:25.870388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:25.870812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:25.910983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:25.913672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:25.937368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:26.007804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 997/1989] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:26.031510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:26.033598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:26.156130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1697db-68ca-4976-8eb9-dd9d2dbf17c4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:26.173302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeebb6-a127-4ac6-a8e2-28e36d1c44a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:26.174364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeebb6-a127-4ac6-a8e2-28e36d1c44a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:26.175663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeebb6-a127-4ac6-a8e2-28e36d1c44a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:26.176187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeebb6-a127-4ac6-a8e2-28e36d1c44a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:26.199859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeebb6-a127-4ac6-a8e2-28e36d1c44a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:26.201905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdeebb6-a127-4ac6-a8e2-28e36d1c44a2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:26.227537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 993/1990] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:26.306709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:26.308314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:26.326596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2988f-72e1-49d7-9b45-ed8fe5185b1b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:26.333245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:26.335500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:26.335823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:26.336097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] PATH_INFO: `/v3/projects/67e855fc220b42a5a5e4d3ed7700b59c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:26.338631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e855fc220b42a5a5e4d3ed7700b59c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:26.356196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:26.468851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e855fc220b42a5a5e4d3ed7700b59c', 'request_id': 'req-5c844a93-a141-44a5-ba39-17a90750369a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.483480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e855fc220b42a5a5e4d3ed7700b59c', 'request_id': 'req-5c844a93-a141-44a5-ba39-17a90750369a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.491270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e855fc220b42a5a5e4d3ed7700b59c', 'request_id': 'req-5c844a93-a141-44a5-ba39-17a90750369a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.512078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] The token cache is being invalidate because project 67e855fc220b42a5a5e4d3ed7700b59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:26.512378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.513210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.516559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 998/1991] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:26 2022] DELETE /identity/v3/projects/67e855fc220b42a5a5e4d3ed7700b59c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:26.521443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed048895-00a6-474b-a37b-8aa78db37022 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:26.522259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed048895-00a6-474b-a37b-8aa78db37022 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:26.522545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed048895-00a6-474b-a37b-8aa78db37022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:26.544700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ed048895-00a6-474b-a37b-8aa78db37022 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:26.546136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ed048895-00a6-474b-a37b-8aa78db37022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:26.659097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:26.660776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:26.669908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 994/1992] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:26.681075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:26.681361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:26.681749 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:26.690393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c844a93-a141-44a5-ba39-17a90750369a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:26.697575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:26.698351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:26.698766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:26.699101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] PATH_INFO: `/v3/projects/cba313d1e2b04c72be4909db2d95e16e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:26.700974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba313d1e2b04c72be4909db2d95e16e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:26.720345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:26.723286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:26.726733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:26.747586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 995/1993] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:26.751080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:26.762465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:26.776533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f40a1e-99a7-42ca-bc4a-15f819fb1e35 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:26.791343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:26.793840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:26.794283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:26.795241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:26.796300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:26.801589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:26.824231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:26.844132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba313d1e2b04c72be4909db2d95e16e', 'request_id': 'req-76573ffb-7bb4-4ef7-8973-0b119ec98e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.865892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba313d1e2b04c72be4909db2d95e16e', 'request_id': 'req-76573ffb-7bb4-4ef7-8973-0b119ec98e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.872120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 996/1994] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:46:26.878470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:26.885709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba313d1e2b04c72be4909db2d95e16e', 'request_id': 'req-76573ffb-7bb4-4ef7-8973-0b119ec98e83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.904171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fd57e-2ec8-49c1-895e-54b2fab5c7fe admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:26.913999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] The token cache is being invalidate because project cba313d1e2b04c72be4909db2d95e16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:26.913999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.918002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:26.919913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 999/1995] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:26 2022] DELETE /identity/v3/projects/cba313d1e2b04c72be4909db2d95e16e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:26.923565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:26.927362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.018934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.019803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.020179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.021145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.022024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.024300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.057675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:27.137189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 997/1996] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:26 2022] POST /identity/v3/projects => generated 363 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:27.138225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.140257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.155080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76573ffb-7bb4-4ef7-8973-0b119ec98e83 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.253088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f387da-a80c-47f4-b698-39779d2100be admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.259096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.259779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.260065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.260331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] PATH_INFO: `/v3/users/6598ccba35b346ea933ec1ceb36db48f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.260839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.261612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6598ccba35b346ea933ec1ceb36db48f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.261929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.262356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.262792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] PATH_INFO: `/v3/users/599dd2c04c5a4b169213c44c6a223e42` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.264049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.264632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=599dd2c04c5a4b169213c44c6a223e42)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.267948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.327641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6598ccba35b346ea933ec1ceb36db48f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.342582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '599dd2c04c5a4b169213c44c6a223e42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.342852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6598ccba35b346ea933ec1ceb36db48f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.353905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6598ccba35b346ea933ec1ceb36db48f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.354921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6598ccba35b346ea933ec1ceb36db48f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.371550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '599dd2c04c5a4b169213c44c6a223e42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.376428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:27.382608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6598ccba35b346ea933ec1ceb36db48f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.398815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 998/1997] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:27 2022] DELETE /identity/v3/users/6598ccba35b346ea933ec1ceb36db48f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:27.399844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.403633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.410972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '599dd2c04c5a4b169213c44c6a223e42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.411995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '599dd2c04c5a4b169213c44c6a223e42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.424203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '599dd2c04c5a4b169213c44c6a223e42'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.454610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1000/1998] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:26 2022] DELETE /identity/v3/users/599dd2c04c5a4b169213c44c6a223e42 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:27.454610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.475798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.499413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aefe38-8538-4501-b9f4-b721f20b9cb9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.594627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:27.602846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c32504-b49a-4ea5-b1b0-64ffcf168a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.603988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32504-b49a-4ea5-b1b0-64ffcf168a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.604428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32504-b49a-4ea5-b1b0-64ffcf168a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.604851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c32504-b49a-4ea5-b1b0-64ffcf168a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.605646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c32504-b49a-4ea5-b1b0-64ffcf168a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.607777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c32504-b49a-4ea5-b1b0-64ffcf168a55 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.612479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a574a7-3f4a-458c-be5a-4e2befeddf65 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.621062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.621855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.622977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.623258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] PATH_INFO: `/v3/users/5eba40ff99644b54a344044be1135d70` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.624756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eba40ff99644b54a344044be1135d70)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.627850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.637461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 999/1999] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:27 2022] POST /identity/v3/users => generated 282 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:27.639445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.639698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.640102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.670197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:27.672478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.684392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1000/2000] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:27.685254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.686559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.701953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eba40ff99644b54a344044be1135d70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.707949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c3b5a-74b2-4bf0-9546-ec63ff176931 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.712382 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eba40ff99644b54a344044be1135d70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.713781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.714490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.714828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.715102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.715853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.717500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.728669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eba40ff99644b54a344044be1135d70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.729596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eba40ff99644b54a344044be1135d70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.733441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1001/2001] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:27.733441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.733881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.746147 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eba40ff99644b54a344044be1135d70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:27.754828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1001/2002] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:27 2022] DELETE /identity/v3/users/5eba40ff99644b54a344044be1135d70 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:27.756357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.758865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.817016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e712323-f1ca-4fc1-b134-0af261afac0c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.837005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc824b-e878-49d9-bebe-bb1ea76be8de admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.842935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafd7d1-57b7-4de1-bb2a-d4fa3c8bc192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.844197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafd7d1-57b7-4de1-bb2a-d4fa3c8bc192 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.844499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafd7d1-57b7-4de1-bb2a-d4fa3c8bc192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.844868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafd7d1-57b7-4de1-bb2a-d4fa3c8bc192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.845754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafd7d1-57b7-4de1-bb2a-d4fa3c8bc192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.847473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafd7d1-57b7-4de1-bb2a-d4fa3c8bc192 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.860334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:27.865359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1002/2003] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:27.867691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.868118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.868876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.887808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.888410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.888702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.888953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.890924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.892335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:27.893631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.904498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1003/2004] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:27.905331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.907514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.909368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.915815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828970c5-8113-4329-9b25-7e3156f9abc2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.922011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.922011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.922286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1002/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:27.923085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.923327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] PATH_INFO: `/v3/projects/771c30a1806443868a248befdcaa3d21/users/6130f3cd53f94107a6f09330e538e7d3/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.923728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.925272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:27.943309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060f7bd-20d1-4e26-af06-a9fbdde672dc service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:27.944719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771c30a1806443868a248befdcaa3d21, user_id=6130f3cd53f94107a6f09330e538e7d3, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.951738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.954279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:27.955259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:27.955690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:27.956203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] PATH_INFO: `/v3/projects/8e1b780581554100bfb6fe1116e6f3eb` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:27.958479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1b780581554100bfb6fe1116e6f3eb)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:27.962946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:27.969844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1004/2006] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:27 2022] PUT /identity/v3/projects/771c30a1806443868a248befdcaa3d21/users/6130f3cd53f94107a6f09330e538e7d3/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:27.970819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:27.972662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.034966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1b780581554100bfb6fe1116e6f3eb', 'request_id': 'req-eaf2ddf1-345f-4104-a533-1cd19c56fdce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.038981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:28.044496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1b780581554100bfb6fe1116e6f3eb', 'request_id': 'req-eaf2ddf1-345f-4104-a533-1cd19c56fdce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.058317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1b780581554100bfb6fe1116e6f3eb', 'request_id': 'req-eaf2ddf1-345f-4104-a533-1cd19c56fdce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.059720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055ef9e-377e-4b0f-8ac7-60b9029c4d83 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:28.081650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] The token cache is being invalidate because project 8e1b780581554100bfb6fe1116e6f3eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:28.082568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.083363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-eaf2ddf1-345f-4104-a533-1cd19c56fdce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.086666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1003/2007] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:27 2022] DELETE /identity/v3/projects/8e1b780581554100bfb6fe1116e6f3eb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:28.093156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa65d4-b446-467f-ab88-597fb108ea31 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.094011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa65d4-b446-467f-ab88-597fb108ea31 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.094011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa65d4-b446-467f-ab88-597fb108ea31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.125596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-26fa65d4-b446-467f-ab88-597fb108ea31 None None] MFA Rules not processed for user `6130f3cd53f94107a6f09330e538e7d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:28.126827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-26fa65d4-b446-467f-ab88-597fb108ea31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.156435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:28.157080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.157495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.157834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.159926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.228702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1004/2008] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:28.228702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd21819-3036-4c69-ba7d-2d1d6a4e6c9b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.228702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd21819-3036-4c69-ba7d-2d1d6a4e6c9b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.228702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd21819-3036-4c69-ba7d-2d1d6a4e6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.260915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-dcd21819-3036-4c69-ba7d-2d1d6a4e6c9b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:28.263337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:28.264939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:28.265219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dcd21819-3036-4c69-ba7d-2d1d6a4e6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.277970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1005/2009] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:28.280326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:28.282226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.284420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1005/2010] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:28.303969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.305768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.306760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.352702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:28.354816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.382999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1006/2011] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:28.382999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:28.411041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.461383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e311a44-ee4e-4326-9df7-544d8916d135 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:28.473000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:28.473841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.474383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.476624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.495392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:28.496066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be67e1-8daa-4d84-a1c0-5f2061dcbb32 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:28.498705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:28.520856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1006/2012] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:28.523514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:28.527113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:28.528256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.528799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.529251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] PATH_INFO: `/v3/projects/ab05b46b3f81484b80daf796469d4db0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.533268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab05b46b3f81484b80daf796469d4db0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:28.539628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:28.558003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd5e9-f55d-4615-b6a4-b1792c5bd9aa service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:28.578110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:28.581380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.583048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.583529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.590836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:28.592751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:28.598788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:28.650371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1007/2013] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:28.652343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:28.705373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:28.722851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0d15e-1d56-4a47-870b-e356d518e10d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:28.744731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab05b46b3f81484b80daf796469d4db0', 'request_id': 'req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.759592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab05b46b3f81484b80daf796469d4db0', 'request_id': 'req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.775995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab05b46b3f81484b80daf796469d4db0', 'request_id': 'req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.777935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:28.806806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] The token cache is being invalidate because project ab05b46b3f81484b80daf796469d4db0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:28.807784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.809363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.811286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:28.811959 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.812648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1007/2014] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:28 2022] DELETE /identity/v3/projects/ab05b46b3f81484b80daf796469d4db0 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:28.813071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.813504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.814208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:28.816177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:28.818405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:28.821476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.845274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c41ee-10c9-4ad0-b3c3-7f5e4c72938f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:28.851003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:28.851734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:28.852308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:28.852590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] PATH_INFO: `/v3/users/5176c22c411548c38fb404bbdb22aa38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:28.854260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5176c22c411548c38fb404bbdb22aa38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:28.856436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:28.884223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1008/2015] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:28 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:28.890581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:28.895668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:28.939398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5176c22c411548c38fb404bbdb22aa38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.951681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5176c22c411548c38fb404bbdb22aa38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.961620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5176c22c411548c38fb404bbdb22aa38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.962336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5176c22c411548c38fb404bbdb22aa38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.983537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5176c22c411548c38fb404bbdb22aa38'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:28.997160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1008/2016] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:28 2022] DELETE /identity/v3/users/5176c22c411548c38fb404bbdb22aa38 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:29.060761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79252532-2847-4ab6-8b2c-3ad06a803223 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.259052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.259052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.259052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.259052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.260305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.260305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.300231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1009/2017] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:28 2022] POST /identity/v3/users => generated 293 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:29.306569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:29.308096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:29.340840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6a4b6-3666-4497-9a6e-60c9befc3924 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.358303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.358303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.358303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.358303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.359580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.359580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.385866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1009/2018] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:29.393052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:29.409770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929ff0d-3193-4d82-aa37-c0e8cd33f0d8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.433219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.433219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.433530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.433685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.434362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.438061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.462912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1010/2019] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:29.467554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:29.483347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb000f-a455-43de-9393-826424bcc3a5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.493815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d5963-7a6c-4a1c-8075-74e2d795d103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.497809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d5963-7a6c-4a1c-8075-74e2d795d103 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.497809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d5963-7a6c-4a1c-8075-74e2d795d103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.497809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d5963-7a6c-4a1c-8075-74e2d795d103 admin admin] PATH_INFO: `/v3/projects/5390ace7f5e74873addd0a0a1877c308/users/29afea8f5a60409a8264c2ee0719cc3f/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.515837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:29.517822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:29.539029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d5963-7a6c-4a1c-8075-74e2d795d103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5390ace7f5e74873addd0a0a1877c308, user_id=29afea8f5a60409a8264c2ee0719cc3f, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.542162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d5963-7a6c-4a1c-8075-74e2d795d103 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.546117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208963cd-a466-40cd-bbff-14f880b19343 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.555845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.556859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.557348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.557803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] PATH_INFO: `/v3/projects/3f4b446b18d148638b974bab990679c8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.560374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f4b446b18d148638b974bab990679c8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.562954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.578241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1010/2020] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:29 2022] PUT /identity/v3/projects/5390ace7f5e74873addd0a0a1877c308/users/29afea8f5a60409a8264c2ee0719cc3f/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:29.602505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.602780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.603311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.651289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] MFA Rules not processed for user `29afea8f5a60409a8264c2ee0719cc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:29.652894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:29.653782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:29.666369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f4b446b18d148638b974bab990679c8', 'request_id': 'req-848d6b7d-882f-46bb-ae57-0e74037ecc6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:29.678896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f4b446b18d148638b974bab990679c8', 'request_id': 'req-848d6b7d-882f-46bb-ae57-0e74037ecc6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:29.687216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f4b446b18d148638b974bab990679c8', 'request_id': 'req-848d6b7d-882f-46bb-ae57-0e74037ecc6b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:29.705133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] The token cache is being invalidate because project 3f4b446b18d148638b974bab990679c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:29.705599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:29.706383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:29.708642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1011/2021] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:29 2022] DELETE /identity/v3/projects/3f4b446b18d148638b974bab990679c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:29.712881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:29.715063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:29.715919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:29.780290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1011/2022] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:29.783256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:29.788509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:29.884912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6b7d-882f-46bb-ae57-0e74037ecc6b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.895963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.899681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.900480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.901489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] PATH_INFO: `/v3/users/8a80f41069d449b0ae736a19c214b702` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.903845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a80f41069d449b0ae736a19c214b702)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.909420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.955571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc0bf9-1e1c-4556-bc8c-c15018a5c531 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:29.966655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:29.967963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:29.968327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:29.968537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:29.969240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:29.971533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:29.998693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a80f41069d449b0ae736a19c214b702'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:30.015808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1012/2023] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:29 2022] POST /identity/v3/projects => generated 387 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:30.018419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a80f41069d449b0ae736a19c214b702'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:30.021260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.042973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a80f41069d449b0ae736a19c214b702'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:30.043846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a80f41069d449b0ae736a19c214b702'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:30.061515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a80f41069d449b0ae736a19c214b702'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:30.079329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1012/2024] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:29 2022] DELETE /identity/v3/users/8a80f41069d449b0ae736a19c214b702 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:30.124774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b29a3-9526-4adb-a1b3-ffaa3d178129 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.221722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.222545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.223624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.224116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.225645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.228484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:30.257665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1013/2025] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:30 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:30.261898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.266520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:30.289695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405370c4-cad0-4ecb-a652-71ea7ef5fa05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.298205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.299308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.299633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.299879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.300973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.302827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:30.321692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1013/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:30.325332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.332480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:30.339376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f531a9-a468-4a85-82ce-9f1336f5cd1c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.346116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05281ad-03a2-4158-9212-ecef612a02c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.347676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05281ad-03a2-4158-9212-ecef612a02c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.347983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05281ad-03a2-4158-9212-ecef612a02c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.348258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05281ad-03a2-4158-9212-ecef612a02c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.348922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05281ad-03a2-4158-9212-ecef612a02c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.351107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05281ad-03a2-4158-9212-ecef612a02c2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:30.368362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.368597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1014/2027] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:30.380428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124a443-667e-4a2e-a948-c52386feda57 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.386084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.386842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.387120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.387387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] PATH_INFO: `/v3/projects/9a0f61dde2b24338bb7b8c57d4b6f274/users/cdbf9a9d440a4c698e90e719cece2fd9/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.407086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0f61dde2b24338bb7b8c57d4b6f274, user_id=cdbf9a9d440a4c698e90e719cece2fd9, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.410200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:30.431971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1014/2028] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:30 2022] PUT /identity/v3/projects/9a0f61dde2b24338bb7b8c57d4b6f274/users/cdbf9a9d440a4c698e90e719cece2fd9/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:30.442413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.442732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.443207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.487123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] MFA Rules not processed for user `cdbf9a9d440a4c698e90e719cece2fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:30.489093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:30.659219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1015/2029] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:30.661328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.700961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:30.804229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eef95c-fdce-4706-b017-a81a0d9b5d46 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.809212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.810606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.811018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.811209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.812031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.814579 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:30.852420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1015/2030] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:30 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:30.858703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.860371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:30.862783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:30.876199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a4ab77-1721-4135-bc30-19a4d5181fa1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.958601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.959317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff1247-e411-4570-8f1f-714221337921 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:30.960361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.960687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.960936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.961737 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.964164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:30.974075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:30.974752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:30.975061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:30.975831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] PATH_INFO: `/v3/projects/41f9f4e01e0446759c3d6b0d0b4fead6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:30.979552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f9f4e01e0446759c3d6b0d0b4fead6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:30.981727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.013487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1016/2031] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:30 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:31.017756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.061684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6cd1-30da-485f-9908-e0f6ae5d5465 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.078857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f9f4e01e0446759c3d6b0d0b4fead6', 'request_id': 'req-ec639c23-2c45-422c-a334-c033bf180831'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.095894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f9f4e01e0446759c3d6b0d0b4fead6', 'request_id': 'req-ec639c23-2c45-422c-a334-c033bf180831'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.106239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f9f4e01e0446759c3d6b0d0b4fead6', 'request_id': 'req-ec639c23-2c45-422c-a334-c033bf180831'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.123900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] The token cache is being invalidate because project 41f9f4e01e0446759c3d6b0d0b4fead6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:31.124209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.124979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec639c23-2c45-422c-a334-c033bf180831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.126878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1016/2032] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:30 2022] DELETE /identity/v3/projects/41f9f4e01e0446759c3d6b0d0b4fead6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:31.137745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7681673-c911-4f7d-9f5d-1ca1a4be3aec None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.138032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7681673-c911-4f7d-9f5d-1ca1a4be3aec None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.138308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7681673-c911-4f7d-9f5d-1ca1a4be3aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.191996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.192480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f7681673-c911-4f7d-9f5d-1ca1a4be3aec None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:31.192622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.192885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.193155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.194058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.195887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f7681673-c911-4f7d-9f5d-1ca1a4be3aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.196060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.212894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1017/2033] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:31.213806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.216001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.224259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1017/2034] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:31.226352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.226698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.227028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.235768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c79b6-6166-423a-84fe-75fe8a12bdd5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.242948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.243535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.243935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.244212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] PATH_INFO: `/v3/projects/81e8d52e5b674335b6546ddf5842024b/users/c98dd1f925214fa5b7e71ea237ad572d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.248863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:31.273423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:31.274178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e8d52e5b674335b6546ddf5842024b, user_id=c98dd1f925214fa5b7e71ea237ad572d, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.274674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.279508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.297507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1018/2035] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:31.298620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.335287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb63a58-af00-42ae-8b39-174734c991fc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.343080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1018/2036] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:31 2022] PUT /identity/v3/projects/81e8d52e5b674335b6546ddf5842024b/users/c98dd1f925214fa5b7e71ea237ad572d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:31.349413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.372334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:31.493786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2749-fa67-4ac4-ab4e-1a42911be8dd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.497142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.497925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.498306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.498682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.499513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:31.499998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.502467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.507098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.508141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.508782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.509240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.510368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.515779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.518287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1019/2037] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:31.520101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.539010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1019/2038] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:31.539918 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.547639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.557299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a93933-7a16-4fc7-9350-e83dc34be8f5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.570875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.573114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.576445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.576774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.577794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d45585-23dc-49ed-b762-7b0e5757d509 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.579559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.583301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.587788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.588378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.588853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.589330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] PATH_INFO: `/v3/projects/c067a61dd8744757a5aa348e702c19ae` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.591474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c067a61dd8744757a5aa348e702c19ae)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.594885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.635355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1020/2039] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:31.636687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.680059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c067a61dd8744757a5aa348e702c19ae', 'request_id': 'req-94b0b147-3968-44ba-a8dd-e557001a19bb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.689354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c067a61dd8744757a5aa348e702c19ae', 'request_id': 'req-94b0b147-3968-44ba-a8dd-e557001a19bb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.696519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c067a61dd8744757a5aa348e702c19ae', 'request_id': 'req-94b0b147-3968-44ba-a8dd-e557001a19bb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.713559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] The token cache is being invalidate because project c067a61dd8744757a5aa348e702c19ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:31.713844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.714695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:31.716358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1020/2040] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:31 2022] DELETE /identity/v3/projects/c067a61dd8744757a5aa348e702c19ae => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:31.717105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.718594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.736710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d177936-603f-491f-a8a0-11d189390c01 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.791249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0b147-3968-44ba-a8dd-e557001a19bb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.796815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1a593-8074-46ee-9846-fe114a66750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.797550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1a593-8074-46ee-9846-fe114a66750e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.797808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1a593-8074-46ee-9846-fe114a66750e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.798077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1a593-8074-46ee-9846-fe114a66750e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.798376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fac6e2-ec2f-4602-905f-c6273b4e5847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.798708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1a593-8074-46ee-9846-fe114a66750e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.799178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fac6e2-ec2f-4602-905f-c6273b4e5847 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.799522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fac6e2-ec2f-4602-905f-c6273b4e5847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.799967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fac6e2-ec2f-4602-905f-c6273b4e5847 admin admin] PATH_INFO: `/v3/projects/81e8d52e5b674335b6546ddf5842024b/users/c98dd1f925214fa5b7e71ea237ad572d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.800809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1a593-8074-46ee-9846-fe114a66750e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.802016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fac6e2-ec2f-4602-905f-c6273b4e5847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e8d52e5b674335b6546ddf5842024b, user_id=c98dd1f925214fa5b7e71ea237ad572d, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.804842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fac6e2-ec2f-4602-905f-c6273b4e5847 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.815006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:31.821816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1021/2041] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:31 2022] PUT /identity/v3/projects/81e8d52e5b674335b6546ddf5842024b/users/c98dd1f925214fa5b7e71ea237ad572d/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:31.823436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.823694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.824037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.835282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1021/2042] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:31.836906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.837193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.837475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.846611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:31.848105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.869415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] MFA Rules not processed for user `c98dd1f925214fa5b7e71ea237ad572d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:31.870557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:31.879014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:31.929479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1022/2043] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:31.929479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.935383 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb67a41-5758-4deb-8f7d-d3819bdbbd6c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.942069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f03f9-d94d-4e33-8756-e1d5e33df797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.942706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f03f9-d94d-4e33-8756-e1d5e33df797 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.942982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f03f9-d94d-4e33-8756-e1d5e33df797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.943252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f03f9-d94d-4e33-8756-e1d5e33df797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.943971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f03f9-d94d-4e33-8756-e1d5e33df797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.945632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f03f9-d94d-4e33-8756-e1d5e33df797 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:31.957661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1023/2044] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:31.959701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1022/2045] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:31.960347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.961019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.961301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.964515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:31.980609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70411508-2e70-4fa3-8f1d-674a2602bc88 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:31.986101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:31.986789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:31.987074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:31.987326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:31.987987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:31.989609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.001511 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1023/2046] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:32.002401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.004908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:32.015307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:32.016657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:32.033942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1024/2047] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:32.034601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.045954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ed1d3-94d8-49dd-a238-018e0fcb0785 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.054269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecf1c6-70e8-4829-9d4d-db5604101d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:32.055074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf1c6-70e8-4829-9d4d-db5604101d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.055864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf1c6-70e8-4829-9d4d-db5604101d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.055864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecf1c6-70e8-4829-9d4d-db5604101d22 admin admin] PATH_INFO: `/v3/projects/305f2c09e967450fa66e89e89dc9ee54/users/314f4f6eff6b48a1b77abab96a021da0/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.082048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecf1c6-70e8-4829-9d4d-db5604101d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=305f2c09e967450fa66e89e89dc9ee54, user_id=314f4f6eff6b48a1b77abab96a021da0, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:32.085151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecf1c6-70e8-4829-9d4d-db5604101d22 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.106518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb96e9f-c34d-4bb4-b487-681f7e85917f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.112585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1025/2048] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:32 2022] PUT /identity/v3/projects/305f2c09e967450fa66e89e89dc9ee54/users/314f4f6eff6b48a1b77abab96a021da0/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:32.114590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0404b-86b2-44b9-b2bc-8835eb787b11 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.115077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0404b-86b2-44b9-b2bc-8835eb787b11 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.115358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0404b-86b2-44b9-b2bc-8835eb787b11 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.117893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1026/2049] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:46:32.118634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0404b-86b2-44b9-b2bc-8835eb787b11 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.183548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207699d-ff52-483d-b608-ecf2e34ac77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:32.184414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207699d-ff52-483d-b608-ecf2e34ac77e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.184702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207699d-ff52-483d-b608-ecf2e34ac77e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.184974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207699d-ff52-483d-b608-ecf2e34ac77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.251634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0404b-86b2-44b9-b2bc-8835eb787b11 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.257026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:32.257985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.258205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.258582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.259302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:32.259664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:32.261438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f49ae-c7c6-4062-9808-cfa1563e37f5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.279093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1027/2050] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:32.280729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.281115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.281503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.305184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207699d-ff52-483d-b608-ecf2e34ac77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:32.306658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207699d-ff52-483d-b608-ecf2e34ac77e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.312472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:32.330676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] MFA Rules not processed for user `314f4f6eff6b48a1b77abab96a021da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:32.332118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:32.339529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1024/2051] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:32.343386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.345083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.345365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.409885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:32.411247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:32.415112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:32.493454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1028/2052] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:32.494414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.505826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215e8d3-2c00-4af0-85b2-c1a1a59caccd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.528456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:32.529256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.529712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.530125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.530921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:32.533318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.601121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1029/2053] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:32 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:32.601121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.650304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1025/2054] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:46:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:32.653139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.682865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaee197-bb99-4d07-ad0d-1d94b1e98ce7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.751493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd5456-2f4f-4c12-b971-7491af11c71f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.768704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:32.769324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.769599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.769864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.770510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:32.775573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.852840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1030/2055] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:32 2022] POST /identity/v3/projects => generated 369 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:32.854103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.859696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:32.903891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:32.904894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:32.905305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:32.905713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:32.906586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:32.908985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:32.967867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1026/2056] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:32 2022] POST /identity/v3/users => generated 291 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:32.969566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:32.987438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f8e8-40ef-4bf5-a0a4-ccc414200506 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:32.992627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c6931-6927-4b3a-9bac-51f221bb6537 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.001556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.002179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.002452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.002717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.021824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.024112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.051089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1031/2057] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:33.052077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.145736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:33.185283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.190450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.190730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.191985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.198667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.209669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.223036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc662200-a675-41d8-a23a-1d4e51bacbff service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.250436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.251081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.251510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.251797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.253081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.255340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.296031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1027/2059] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:32 2022] POST /identity/v3/users => generated 291 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:33.298793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1032/2059] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:33.299210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.302115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.324028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73073dcd-a1c8-4762-aea0-39b9a84deb90 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.330621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.331255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.331574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.331942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.332668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.335162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.349107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28f0667-1586-4321-8aa6-6d7bd89bc79f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.353432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1033/2060] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:33.355128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.369747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c976327-12a8-49fd-84b7-834049dadaa6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.376579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.377297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.377571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.377911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] PATH_INFO: `/v3/projects/b5e31e5419ab4b7baf62b46a7d1ae3a5/users/18ea92178b0f45fb987ae625b6b05390/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.384864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.386789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.387487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.388799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.391170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.397898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.412212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:33.425152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e31e5419ab4b7baf62b46a7d1ae3a5, user_id=18ea92178b0f45fb987ae625b6b05390, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.431662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1028/2061] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:33.432898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.453851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8ed223-15a9-4782-81cf-628666c6744e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.464319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.470738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.479646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.480141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.480420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] PATH_INFO: `/v3/projects/32d8c5f0c61b4064a5194137b6f5b320/users/592b7c0935b44705b61567e3ab93ccdd/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.510441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1034/2062] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:33 2022] PUT /identity/v3/projects/b5e31e5419ab4b7baf62b46a7d1ae3a5/users/18ea92178b0f45fb987ae625b6b05390/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:33.516069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.524842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d8c5f0c61b4064a5194137b6f5b320, user_id=592b7c0935b44705b61567e3ab93ccdd, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.532516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.575179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1029/2063] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:33 2022] PUT /identity/v3/projects/32d8c5f0c61b4064a5194137b6f5b320/users/592b7c0935b44705b61567e3ab93ccdd/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:33.577026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.579001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:33.666616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c14783-0dfc-482c-8670-476566346226 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.730493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994e76b-715d-4f74-b217-3c705a1f33ff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.735966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.736664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.737735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.738168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.745359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:33.839048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46090c-7da6-4e96-bfbd-dcf076d8ad90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.839848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46090c-7da6-4e96-bfbd-dcf076d8ad90 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.840135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46090c-7da6-4e96-bfbd-dcf076d8ad90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.840411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46090c-7da6-4e96-bfbd-dcf076d8ad90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.841252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46090c-7da6-4e96-bfbd-dcf076d8ad90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.844191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46090c-7da6-4e96-bfbd-dcf076d8ad90 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.866998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1035/2064] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:33 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:33.868362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.868661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.869321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.894482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.896521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.918885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1030/2065] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:33.928337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] MFA Rules not processed for user `592b7c0935b44705b61567e3ab93ccdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:33.928337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:33.928869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:33.948661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998291c1-36ff-425d-922d-a0ba26370f54 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:33.955019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03204-c895-4131-a7ac-d57eea285e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:33.956102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03204-c895-4131-a7ac-d57eea285e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.956558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03204-c895-4131-a7ac-d57eea285e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.956982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03204-c895-4131-a7ac-d57eea285e90 admin admin] PATH_INFO: `/v3/projects/b5e31e5419ab4b7baf62b46a7d1ae3a5/users/18ea92178b0f45fb987ae625b6b05390/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:33.961652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d03204-c895-4131-a7ac-d57eea285e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e31e5419ab4b7baf62b46a7d1ae3a5, user_id=18ea92178b0f45fb987ae625b6b05390, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:33.965009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d03204-c895-4131-a7ac-d57eea285e90 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:33.966736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:33.981058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1031/2066] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:33 2022] PUT /identity/v3/projects/b5e31e5419ab4b7baf62b46a7d1ae3a5/users/18ea92178b0f45fb987ae625b6b05390/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:33.995662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:33.995973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:33.996258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.040089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] MFA Rules not processed for user `18ea92178b0f45fb987ae625b6b05390`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:34.041492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:34.065689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1036/2067] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:34.071184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.102727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230204a-d6d4-41c0-8595-219881a8b5ca None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.248877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.250564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.250897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.251355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.252414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.254629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.316869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1037/2068] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:34 2022] POST /identity/v3/projects => generated 383 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:34.322983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.333711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1032/2069] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:34.426754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.431831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:34.445561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a288549-8704-4451-9468-b38707ed701c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.456145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.461592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.462370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.462982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.464890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.470096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.519402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1038/2070] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:34 2022] POST /identity/v3/users => generated 298 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:34.520670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.547146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9980a8-fdde-4f4a-a955-80af8a4e55e2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.563501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.563971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.564423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.564905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.565815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.572402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.602414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6eb0d-bdee-4c41-95fd-b17829c8203b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.606055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1039/2071] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:34.611181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.636931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39089f36-b611-40fb-a5c2-d1f934a0d4b1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.650023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.650906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.651782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.651782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] PATH_INFO: `/v3/projects/8df73302429f4f98aea14f40862afa0a/users/4eb5e06b2465471399500b16c14c2e98/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.677689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df73302429f4f98aea14f40862afa0a, user_id=4eb5e06b2465471399500b16c14c2e98, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.681475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.706138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1040/2072] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:34 2022] PUT /identity/v3/projects/8df73302429f4f98aea14f40862afa0a/users/4eb5e06b2465471399500b16c14c2e98/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:34.707616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.722197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:34.742656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.743954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.744397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.744673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.859447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be5aac-92be-45ad-81a4-70a9d356a9a1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.869851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.870738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.871345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.871860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.872922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.875783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.889070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.890865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.896947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1041/2073] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:34 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:34.904183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:34.912105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1033/2074] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:34.935285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e1d8f-ea9c-4c06-8f98-8286d6b3602e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:34.948304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3cf2a-50b3-4d67-98d2-5861539a8cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:34.949375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3cf2a-50b3-4d67-98d2-5861539a8cea admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:34.950082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3cf2a-50b3-4d67-98d2-5861539a8cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:34.950528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3cf2a-50b3-4d67-98d2-5861539a8cea admin admin] PATH_INFO: `/v3/projects/8df73302429f4f98aea14f40862afa0a/users/4eb5e06b2465471399500b16c14c2e98/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:34.953206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3cf2a-50b3-4d67-98d2-5861539a8cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df73302429f4f98aea14f40862afa0a, user_id=4eb5e06b2465471399500b16c14c2e98, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:34.963768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3cf2a-50b3-4d67-98d2-5861539a8cea admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:34.997436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:35.000212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1042/2075] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:34 2022] PUT /identity/v3/projects/8df73302429f4f98aea14f40862afa0a/users/4eb5e06b2465471399500b16c14c2e98/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:35.004546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:35.004848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:35.005128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:35.014025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d068b-9554-4e43-abda-4df04ab8c888 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:35.051917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] MFA Rules not processed for user `4eb5e06b2465471399500b16c14c2e98`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:35.053771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:35.144784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:35.145422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:35.145791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:35.146304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:35.177097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1043/2076] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:35.183141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:35.239505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:35.241815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:35.249566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:35.257599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1034/2077] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:35.260061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b213c-c766-4e56-b83e-88335980bb93 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:35.264125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:35.265305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:35.265586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:35.265847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:35.281477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:35.283349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:35.305179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1044/2078] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:35.432472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:35.446512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd73ee-cb58-4b39-99dc-2d4e2b44194c service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:35.452159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:35.453301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:35.453610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:35.454069 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:35.539981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:35.542449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:35.559916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1035/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:39.010110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:39.178063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172f64f-deb7-4603-b86f-780fab1103d9 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:39.191832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:39.192608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:39.192920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:39.193237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] PATH_INFO: `/v3/users/314f4f6eff6b48a1b77abab96a021da0` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:39.194666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=314f4f6eff6b48a1b77abab96a021da0)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:39.198120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:39.202828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:39.313408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314f4f6eff6b48a1b77abab96a021da0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.315104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:39.336091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314f4f6eff6b48a1b77abab96a021da0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.340584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5d8704-c2a0-4c99-ad56-58c6c12d9812 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:39.347326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314f4f6eff6b48a1b77abab96a021da0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.349725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314f4f6eff6b48a1b77abab96a021da0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.369112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:39.372297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:39.373283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:39.373769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] PATH_INFO: `/v3/users/592b7c0935b44705b61567e3ab93ccdd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:39.380007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592b7c0935b44705b61567e3ab93ccdd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:39.382984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2dc4984b-4636-4d08-b73f-b5abf60b3cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314f4f6eff6b48a1b77abab96a021da0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.386331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:39.391326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1045/2080] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:39 2022] DELETE /identity/v3/users/314f4f6eff6b48a1b77abab96a021da0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:39.407873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56b27-5440-411e-904e-5c2673a483e9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:39.408160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56b27-5440-411e-904e-5c2673a483e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:39.408435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f56b27-5440-411e-904e-5c2673a483e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:39.453958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f2f56b27-5440-411e-904e-5c2673a483e9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:39.455508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f2f56b27-5440-411e-904e-5c2673a483e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:39.498421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592b7c0935b44705b61567e3ab93ccdd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.513564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592b7c0935b44705b61567e3ab93ccdd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.550941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592b7c0935b44705b61567e3ab93ccdd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.552000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592b7c0935b44705b61567e3ab93ccdd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.560703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592b7c0935b44705b61567e3ab93ccdd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:39.569651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1036/2081] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:39 2022] DELETE /identity/v3/users/592b7c0935b44705b61567e3ab93ccdd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:39.570921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:39.572954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:39.586946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1046/2082] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:39.593882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:39.594172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:39.594449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:39.648818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:39.656218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:39.662902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:39.710907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4224d4-6c0b-490d-8d8a-627a227f6218 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:39.719236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:39.720211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:39.720672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:39.721088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:39.739059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:39.866552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:39.869400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1047/2083] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:39.869628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:39.877317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:39.885025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:39.891880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1037/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:39.893196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:39.897227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:39.934904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:40.023447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6351a337-1678-4867-bed3-fe26cbbc5f0b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:40.027713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:40.028380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:40.028657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:40.028931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:40.042379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:40.044469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:40.058979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1048/2085] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:40.098194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3f2e5-dd2b-49fc-98f9-9609096b1c47 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:40.118419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:40.119068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:40.119344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:40.119630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:40.154198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:40.155774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:40.178996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1038/2086] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:40.840359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:40.843544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:40.884741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a326-b0fc-4d11-9027-91f40da99adf service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:40.890200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:40.891699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:40.891896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:40.892151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:40.894607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:40.921552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:40.924394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:40.953987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a93cf4-4dbb-41e9-a73e-39be91df791f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:40.966737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:40.967390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:40.967669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:40.968354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] PATH_INFO: `/v3/projects/305f2c09e967450fa66e89e89dc9ee54` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:40.980591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] RBAC: Authorizing `identity:delete_project(project_id=305f2c09e967450fa66e89e89dc9ee54)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:40.987677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:41.037325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:41.039943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:41.056510 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1049/2087] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:41.057375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:41.058831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:41.079408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ff110-25e5-485a-afc1-7085d922d8c9 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:41.098801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '305f2c09e967450fa66e89e89dc9ee54', 'request_id': 'req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.118605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '305f2c09e967450fa66e89e89dc9ee54', 'request_id': 'req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.127580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '305f2c09e967450fa66e89e89dc9ee54', 'request_id': 'req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.160738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] The token cache is being invalidate because project 305f2c09e967450fa66e89e89dc9ee54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:41.161531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.163071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.172795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1039/2088] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:40 2022] DELETE /identity/v3/projects/305f2c09e967450fa66e89e89dc9ee54 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:41.175170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:41.179542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:41.238429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:41.268989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:41.270929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:41.272018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:41.272519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] PATH_INFO: `/v3/projects/32d8c5f0c61b4064a5194137b6f5b320` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:41.274270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d8c5f0c61b4064a5194137b6f5b320)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:41.281073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:41.426815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d8c5f0c61b4064a5194137b6f5b320', 'request_id': 'req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.445837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852a1c0-3cb5-4cb8-bd77-b9c9dd5c5ded admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:41.459152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d8c5f0c61b4064a5194137b6f5b320', 'request_id': 'req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.463674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:41.494255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d8c5f0c61b4064a5194137b6f5b320', 'request_id': 'req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.520137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] The token cache is being invalidate because project 32d8c5f0c61b4064a5194137b6f5b320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:41.520744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.522018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.525113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1050/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:41 2022] DELETE /identity/v3/projects/32d8c5f0c61b4064a5194137b6f5b320 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:41.528857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:41.531182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:41.595355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:41.596107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:41.596431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:41.597284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] PATH_INFO: `/v3/users/18ea92178b0f45fb987ae625b6b05390` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:41.598913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ea92178b0f45fb987ae625b6b05390)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:41.601937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:41.660966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d1069b-b7f5-47f5-8982-a0a31f0b5b33 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:41.669728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:41.670370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:41.671113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:41.671113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] PATH_INFO: `/v3/users/4eb5e06b2465471399500b16c14c2e98` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:41.672905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb5e06b2465471399500b16c14c2e98)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:41.674817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:41.692042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ea92178b0f45fb987ae625b6b05390'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.701954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ea92178b0f45fb987ae625b6b05390'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.721912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ea92178b0f45fb987ae625b6b05390'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.722824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ea92178b0f45fb987ae625b6b05390'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.735134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-37788069-bb24-4b24-9682-b6bc0138ba9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ea92178b0f45fb987ae625b6b05390'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.744393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1040/2090] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:41 2022] DELETE /identity/v3/users/18ea92178b0f45fb987ae625b6b05390 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:41.758270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb5e06b2465471399500b16c14c2e98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.782237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb5e06b2465471399500b16c14c2e98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.796370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb5e06b2465471399500b16c14c2e98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.797213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb5e06b2465471399500b16c14c2e98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.813038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb5e06b2465471399500b16c14c2e98'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:41.824964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1051/2091] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:41 2022] DELETE /identity/v3/users/4eb5e06b2465471399500b16c14c2e98 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:42.286067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac299d-51a3-43b9-906b-65f9073583c6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.288621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac299d-51a3-43b9-906b-65f9073583c6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.289058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac299d-51a3-43b9-906b-65f9073583c6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.292040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1041/2092] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:46:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:46:42.300459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:42.302012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.321537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.322062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.322707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.352341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:42.354798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.420221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:42.438446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa750c-c802-467c-a769-e97ad31dfc96 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:42.445978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:42.446634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.446909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.447178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] PATH_INFO: `/v3/projects/b5e31e5419ab4b7baf62b46a7d1ae3a5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.448428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e31e5419ab4b7baf62b46a7d1ae3a5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:42.451528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:42.456754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1042/2093] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:46:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:42.457884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:42.459912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.465323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:42.486583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd42f7-1253-4c2a-8a4b-630e084b8d50 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:42.504577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:42.505234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.505503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.505769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] PATH_INFO: `/v3/users/5e01f518dbd947bfafa6aecdccd9cfd5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.531828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e01f518dbd947bfafa6aecdccd9cfd5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:42.537104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:42.556405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e31e5419ab4b7baf62b46a7d1ae3a5', 'request_id': 'req-d470faa9-eec3-431b-b984-63368a06df4e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.572207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e31e5419ab4b7baf62b46a7d1ae3a5', 'request_id': 'req-d470faa9-eec3-431b-b984-63368a06df4e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.579194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e31e5419ab4b7baf62b46a7d1ae3a5', 'request_id': 'req-d470faa9-eec3-431b-b984-63368a06df4e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.608931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] The token cache is being invalidate because project b5e31e5419ab4b7baf62b46a7d1ae3a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:42.609594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.610775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.613779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1052/2094] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:42 2022] DELETE /identity/v3/projects/b5e31e5419ab4b7baf62b46a7d1ae3a5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:42.614378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:42.615874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.662398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e01f518dbd947bfafa6aecdccd9cfd5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.703437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e01f518dbd947bfafa6aecdccd9cfd5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.725284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e01f518dbd947bfafa6aecdccd9cfd5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.726171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e01f518dbd947bfafa6aecdccd9cfd5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.736603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e01f518dbd947bfafa6aecdccd9cfd5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.747999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1043/2095] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:42 2022] DELETE /identity/v3/users/5e01f518dbd947bfafa6aecdccd9cfd5 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:42.749101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:42.751881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.793970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470faa9-eec3-431b-b984-63368a06df4e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:42.852253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c407dd3-8e66-4acc-bace-735889655223 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:42.866406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:42.867341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.867776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.868475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] PATH_INFO: `/v3/projects/8df73302429f4f98aea14f40862afa0a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.871807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df73302429f4f98aea14f40862afa0a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:42.876924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:42.913967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:42.914627 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.914897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.915170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.921230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.935403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:42.937732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86decb-6bb2-4273-b595-5672f650f1a3 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:42.949811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1053/2096] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:42.951153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:42.951536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:42.952117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:42.953940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df73302429f4f98aea14f40862afa0a', 'request_id': 'req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.956095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:42.969644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df73302429f4f98aea14f40862afa0a', 'request_id': 'req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.984523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df73302429f4f98aea14f40862afa0a', 'request_id': 'req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:42.985603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:42.986881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:42.998526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] The token cache is being invalidate because project 8df73302429f4f98aea14f40862afa0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:42.998834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.000163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6e05c0a9-30ba-428d-ba94-1b0afefeba7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.002385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1044/2097] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:42 2022] DELETE /identity/v3/projects/8df73302429f4f98aea14f40862afa0a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:43.011562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27826c8-5e3f-466d-b407-f578a4f9aa38 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.011562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27826c8-5e3f-466d-b407-f578a4f9aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.011562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27826c8-5e3f-466d-b407-f578a4f9aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.032259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:43.046760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a27826c8-5e3f-466d-b407-f578a4f9aa38 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:43.048260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a27826c8-5e3f-466d-b407-f578a4f9aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.083629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1054/2098] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.085076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.086407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.101990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1584c1-7b7c-47eb-887c-a963850d2a82 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.108752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.111222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.111222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.111222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] PATH_INFO: `/v3/projects/981def9b53d4448abde1baf030e5c63c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.119315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=981def9b53d4448abde1baf030e5c63c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.119315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.128831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1045/2099] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.130368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690504-2371-4edf-801a-ca7bd9b106b5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.130733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690504-2371-4edf-801a-ca7bd9b106b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.131691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14690504-2371-4edf-801a-ca7bd9b106b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.154130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-14690504-2371-4edf-801a-ca7bd9b106b5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:43.155523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-14690504-2371-4edf-801a-ca7bd9b106b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.164561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1046/2100] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.165816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20a60-c4d4-4e68-b7c6-18b3313cfc3d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.166094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20a60-c4d4-4e68-b7c6-18b3313cfc3d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.166443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f20a60-c4d4-4e68-b7c6-18b3313cfc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.182147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981def9b53d4448abde1baf030e5c63c', 'request_id': 'req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.189490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-70f20a60-c4d4-4e68-b7c6-18b3313cfc3d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:43.190698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-70f20a60-c4d4-4e68-b7c6-18b3313cfc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.193222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981def9b53d4448abde1baf030e5c63c', 'request_id': 'req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.202679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981def9b53d4448abde1baf030e5c63c', 'request_id': 'req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.227832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] The token cache is being invalidate because project 981def9b53d4448abde1baf030e5c63c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:43.228132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.229101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.230984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1055/2101] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:43 2022] DELETE /identity/v3/projects/981def9b53d4448abde1baf030e5c63c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:43.233369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.235615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.266767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1047/2102] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.268963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.269266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.269775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.293044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:43.295257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.304040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1048/2103] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.304795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.317341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf7bb4-2d47-455e-93e7-10a14ff17951 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.323462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.324940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.325386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.325700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.326360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:43.326659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.328211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24270a64-8c43-4e3a-ab74-62a9f1bd2c62 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.328634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.333651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.334910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.335190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.335461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.336140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:43.336441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.338016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3cc0c-cad7-4e64-84c4-810d15d75c47 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.341685 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1049/2104] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:43.342502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.344157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.357204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1056/2105] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:43.357204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.357204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.357204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.357586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bac5a6-d718-4013-8a69-0a05788eb0e9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.362218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.362831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.363107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.363380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] PATH_INFO: `/v3/users/29afea8f5a60409a8264c2ee0719cc3f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.364528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29afea8f5a60409a8264c2ee0719cc3f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.366475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.375231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:43.376444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.384975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1057/2106] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.385981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.395331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12f243-c592-4393-8ad3-d06c2ab40a4e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.401089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.401942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.402356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.402552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.403285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.405332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.416900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29afea8f5a60409a8264c2ee0719cc3f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.418697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=1,keystone:DELETE=10,keystone:SELECT=89 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:43.426158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29afea8f5a60409a8264c2ee0719cc3f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.436993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1058/2107] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:43.438131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.442743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29afea8f5a60409a8264c2ee0719cc3f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.444352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29afea8f5a60409a8264c2ee0719cc3f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.451790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29afea8f5a60409a8264c2ee0719cc3f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:43.458044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:43.459859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1050/2108] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:43 2022] DELETE /identity/v3/users/29afea8f5a60409a8264c2ee0719cc3f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:43.460921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.462445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.505379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a444b74-770c-4887-a04e-3348581076a9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.530775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b78e12-16f1-499c-9a1a-d18db673192d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.535477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.536114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.536391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.536662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.537387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:43.537701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.539366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.550770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1051/2109] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:43.551643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.553876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.566105 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.566725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.566967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0be7d8-8bf6-452d-85e3-2ab6be156796 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.567064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.567260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.567930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.570426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.572296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d39cd66-2790-4964-81da-499a3021aa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.572898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39cd66-2790-4964-81da-499a3021aa14 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.573177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39cd66-2790-4964-81da-499a3021aa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.573447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d39cd66-2790-4964-81da-499a3021aa14 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.574068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39cd66-2790-4964-81da-499a3021aa14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.576028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d39cd66-2790-4964-81da-499a3021aa14 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.599342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1052/2110] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/users => generated 282 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:43.600678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.600966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.601239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.605113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1059/2111] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/projects => generated 381 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:43.605113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.621840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:43.623070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.671536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf03913-b2d9-4c9d-b9d1-3a7b46b894b6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.676291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.676894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.677302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.677581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.678745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.680673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.686457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1053/2112] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:43.688058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.689387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.703119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef486dd1-c0e8-4d8f-b11d-aeb3ac0df45c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.711778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.712400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.712682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.712953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.713580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.715516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.725538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1060/2113] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/projects => generated 393 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:43.726798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.749970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1054/2114] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/users => generated 291 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:43.750902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.754657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:43.777037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:43.791693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=7,keystone:DELETE=3,keystone:SELECT=90 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:43.807186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a055a0-df08-46f4-b3e0-30f8e310dc30 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.813382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.814159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.814503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.814844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.815617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.817393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.836833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1061/2115] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:43.836833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.838377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad760322-e579-4f41-8bd4-26ccab5f3a24 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.848051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915b6d4-2cfe-4646-a847-3ab3e6b7ec9a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.854984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.855955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.856384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.856823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.857605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.859916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.892966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1062/2116] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:43 2022] POST /identity/v3/users => generated 297 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:43.892966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.903145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374d65f-e1bc-44b6-a18f-edc09220d9df admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.907387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.908040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.908307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.908585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.909396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.910428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.910891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.911354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.913035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.914957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.930338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1063/2117] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:43.930507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.931534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.932170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.943249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1055/2118] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:43.943923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5d0a0-0d28-409f-9a99-c81cf084561a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.944253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.953069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.954017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.954411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf1a1-12a3-4966-ba10-444864f3bd6a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.954715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.955185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.956060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.958870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.959376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.960005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.960277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.960668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.961304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.963036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:43.973827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1064/2119] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:43.975446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.975879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1056/2120] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:43.978784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:43.989398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715131f-1dbc-4d7b-b145-287529ef2c50 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.991307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cc5b7-5711-4e0d-bda7-84b03138697e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:43.997563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.997794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:43.998341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.998604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.998885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:43.999079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:43.999362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:43.999559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:43.999633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] PATH_INFO: `/v3/projects/110eba3edd4b493fb4c1cd83a32986c5/users/ecd52acf2a3747f498635880604c8219/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.001496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.016307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1065/2121] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:44.017462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.018676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=110eba3edd4b493fb4c1cd83a32986c5, user_id=ecd52acf2a3747f498635880604c8219, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.021673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.030360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebecf32-c6cf-4e55-929f-35b53d80f9c5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.040594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398e433-7ed5-4827-acb0-53c6135f6e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.041599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398e433-7ed5-4827-acb0-53c6135f6e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.041896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398e433-7ed5-4827-acb0-53c6135f6e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.042238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398e433-7ed5-4827-acb0-53c6135f6e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.042894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398e433-7ed5-4827-acb0-53c6135f6e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.045775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398e433-7ed5-4827-acb0-53c6135f6e2f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.050552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1057/2122] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:43 2022] PUT /identity/v3/projects/110eba3edd4b493fb4c1cd83a32986c5/users/ecd52acf2a3747f498635880604c8219/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:44.051554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.063783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1066/2123] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:44.065297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.065589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.065868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.106299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] MFA Rules not processed for user `ecd52acf2a3747f498635880604c8219`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:44.107878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:44.170386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb06ac4-3a57-43c1-88ad-7dcd735f0fb2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.175642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.176452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.176795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.177056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] PATH_INFO: `/v3/projects/cb1ea233e49e4827a3848c613a2b9d46/users/42bfc5626ee649d1bfc6472f87d94943/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.207964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1ea233e49e4827a3848c613a2b9d46, user_id=42bfc5626ee649d1bfc6472f87d94943, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.211335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.214728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1067/2124] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:44.215694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.232855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff50ba-0487-427b-a827-3caab57f387a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.241792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1058/2125] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:44 2022] PUT /identity/v3/projects/cb1ea233e49e4827a3848c613a2b9d46/users/42bfc5626ee649d1bfc6472f87d94943/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:44.242220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.306494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba96c9b-b422-4228-b78c-6dc5c89a18ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.307850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba96c9b-b422-4228-b78c-6dc5c89a18ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.308221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba96c9b-b422-4228-b78c-6dc5c89a18ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.308708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba96c9b-b422-4228-b78c-6dc5c89a18ba admin admin] PATH_INFO: `/v3/projects/234f69f21d7a456a9ab6a86c4ac2ede2/users/cfe6eb38e8df492daaef5755ddea7773/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.311592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:44.329857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:44.355196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba96c9b-b422-4228-b78c-6dc5c89a18ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=234f69f21d7a456a9ab6a86c4ac2ede2, user_id=cfe6eb38e8df492daaef5755ddea7773, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.366982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba96c9b-b422-4228-b78c-6dc5c89a18ba admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.410843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1068/2126] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:44 2022] PUT /identity/v3/projects/234f69f21d7a456a9ab6a86c4ac2ede2/users/cfe6eb38e8df492daaef5755ddea7773/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:44.416125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.416602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.417077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.448054 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f13a123-e180-4931-85cb-536bea170db7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.498710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] MFA Rules not processed for user `42bfc5626ee649d1bfc6472f87d94943`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:44.502531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:44.555011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64b43e-129f-43c7-b34c-bb351019aebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.555932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64b43e-129f-43c7-b34c-bb351019aebd admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.556239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64b43e-129f-43c7-b34c-bb351019aebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.556518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64b43e-129f-43c7-b34c-bb351019aebd admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.557177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64b43e-129f-43c7-b34c-bb351019aebd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.559172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64b43e-129f-43c7-b34c-bb351019aebd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.593104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1059/2127] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:44 2022] POST /identity/v3/projects => generated 365 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:44.594508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.594791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.595065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.609441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1069/2128] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:44.612689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.615041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:44.618405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:44.620043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:44.630357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e896b47-0714-4041-9417-1c571659a1f7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.703983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1060/2129] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:44.704289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.719052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.719798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.720116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.720480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] PATH_INFO: `/v3/projects/5390ace7f5e74873addd0a0a1877c308` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.722091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5390ace7f5e74873addd0a0a1877c308)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.722269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723a08-5c58-45e2-b2a6-7cfed140dc9c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.724732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.730093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.730809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.731090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.731545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.732727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.737220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.768753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1061/2130] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:44 2022] POST /identity/v3/users => generated 289 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:44.770549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.784026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9a91-9393-4c31-bcbf-61485a483ad2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:44.797667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5390ace7f5e74873addd0a0a1877c308', 'request_id': 'req-b4822ac2-ae92-4642-9161-febfa3676ab5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:44.809450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5390ace7f5e74873addd0a0a1877c308', 'request_id': 'req-b4822ac2-ae92-4642-9161-febfa3676ab5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:44.818336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5390ace7f5e74873addd0a0a1877c308', 'request_id': 'req-b4822ac2-ae92-4642-9161-febfa3676ab5'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:44.835846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] The token cache is being invalidate because project 5390ace7f5e74873addd0a0a1877c308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:44.836528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:44.837235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:44.838897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1070/2131] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:44 2022] DELETE /identity/v3/projects/5390ace7f5e74873addd0a0a1877c308 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:44.840019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.843206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:44.844387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:44.850374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:44.863385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:44.864844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:44.865926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:44.866386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:44.867166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:44.870030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:44.912075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1062/2132] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:44 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:44.912075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:44.914840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:44.933978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4822ac2-ae92-4642-9161-febfa3676ab5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.037294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.039040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.039426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.039779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.050178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.066870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0daabb-a25d-4226-9bec-1e25d168dbd7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.077400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.079661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.082504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.082504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.082504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.082504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.085698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.091231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.103836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1071/2133] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:45.106255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.107959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.138978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab95ccb-cffa-4642-8d9a-bafdc17b2104 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.161124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.161970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.162975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.163349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] PATH_INFO: `/v3/users/cdbf9a9d440a4c698e90e719cece2fd9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.164695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbf9a9d440a4c698e90e719cece2fd9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.167114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.171560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1063/2134] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:45.200512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.212476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.233956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93404ff-6542-4441-999e-582bd0f28c92 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.244894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.244894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.244894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.244894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.245356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.245779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.273254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbf9a9d440a4c698e90e719cece2fd9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:45.275799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1064/2135] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:45 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:45.277078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.284766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbf9a9d440a4c698e90e719cece2fd9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:45.298785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b6a6f-e2e8-463b-a96d-85bea14f6e64 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.305572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbf9a9d440a4c698e90e719cece2fd9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:45.305815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.306412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbf9a9d440a4c698e90e719cece2fd9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:45.306554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.308495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.312185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] PATH_INFO: `/v3/projects/3b23b74cc4a44c389008f2961ad9a3d9/users/2872d2988f4f49b1b2f7d208769998f0/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.327282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbf9a9d440a4c698e90e719cece2fd9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:45.343607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1072/2136] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:45 2022] DELETE /identity/v3/users/cdbf9a9d440a4c698e90e719cece2fd9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:45.343607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.343944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.347346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b23b74cc4a44c389008f2961ad9a3d9, user_id=2872d2988f4f49b1b2f7d208769998f0, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.350743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.372684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1065/2137] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:45 2022] PUT /identity/v3/projects/3b23b74cc4a44c389008f2961ad9a3d9/users/2872d2988f4f49b1b2f7d208769998f0/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:45.377328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.379231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.413413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:45.493243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c09b2-dddc-4501-bcbd-8c688f842b60 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.497645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482236d-b963-4d26-888c-0de91d6e8f18 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.504927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.507682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.507682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.507682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.509961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.510764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.512696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.513851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.514341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.514667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.515434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.520130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.549092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1073/2139] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:45.550120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.550619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1066/2138] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:45.551936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.589219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5bb73d-3fc7-409c-b9e3-f65c62c944ba admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.591987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3391b5-f537-47cb-8b54-42e43c1dbcf9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.594879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.595652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.596046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.596450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] PATH_INFO: `/v3/projects/f20070b13b534d73b0977b50686fcc3a/users/0ac69ac0663d425a9e1beeb3f48f8c46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.597686 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be6bc9-5b46-4f4b-8011-deff4fc7e797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.598410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6bc9-5b46-4f4b-8011-deff4fc7e797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.598795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6bc9-5b46-4f4b-8011-deff4fc7e797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.599133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6bc9-5b46-4f4b-8011-deff4fc7e797 admin admin] PATH_INFO: `/v3/projects/3b23b74cc4a44c389008f2961ad9a3d9/users/2872d2988f4f49b1b2f7d208769998f0/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.602498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6bc9-5b46-4f4b-8011-deff4fc7e797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b23b74cc4a44c389008f2961ad9a3d9, user_id=2872d2988f4f49b1b2f7d208769998f0, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.606279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6bc9-5b46-4f4b-8011-deff4fc7e797 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.630398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20070b13b534d73b0977b50686fcc3a, user_id=0ac69ac0663d425a9e1beeb3f48f8c46, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.634087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.641203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1067/2140] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:45 2022] PUT /identity/v3/projects/3b23b74cc4a44c389008f2961ad9a3d9/users/2872d2988f4f49b1b2f7d208769998f0/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:45.643106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2771a-61e7-4c50-99cf-9ba9d06ccca9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.643699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2771a-61e7-4c50-99cf-9ba9d06ccca9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.644041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2771a-61e7-4c50-99cf-9ba9d06ccca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.666501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1074/2141] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:45 2022] PUT /identity/v3/projects/f20070b13b534d73b0977b50686fcc3a/users/0ac69ac0663d425a9e1beeb3f48f8c46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:45.667679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.757092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:45.763072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1e667-cc63-4f15-b6ad-07b61fb08f48 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.769856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.770159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.770436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.770701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.771344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.773916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.777470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ddd2771a-61e7-4c50-99cf-9ba9d06ccca9 None None] MFA Rules not processed for user `2872d2988f4f49b1b2f7d208769998f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:45.782691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ddd2771a-61e7-4c50-99cf-9ba9d06ccca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.791049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1075/2142] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:45.792457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.805852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521a3e1-eff4-4251-be85-58546f8903c9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.813139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.814259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.814712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.814976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] PATH_INFO: `/v3/projects/f20070b13b534d73b0977b50686fcc3a/users/0ac69ac0663d425a9e1beeb3f48f8c46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.817424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20070b13b534d73b0977b50686fcc3a, user_id=0ac69ac0663d425a9e1beeb3f48f8c46, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.820681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.839791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1076/2143] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:45 2022] PUT /identity/v3/projects/f20070b13b534d73b0977b50686fcc3a/users/0ac69ac0663d425a9e1beeb3f48f8c46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:45.852863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:45.858917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.912605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1068/2144] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:45.914075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.914457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.915144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.941771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf168434-e65f-4415-b609-9080303049ba admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:45.947166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84077075-ae80-498b-900f-50be65457036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:45.947830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84077075-ae80-498b-900f-50be65457036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:45.948115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84077075-ae80-498b-900f-50be65457036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:45.948874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84077075-ae80-498b-900f-50be65457036 admin admin] PATH_INFO: `/v3/projects/9a0f61dde2b24338bb7b8c57d4b6f274` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:45.949644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84077075-ae80-498b-900f-50be65457036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0f61dde2b24338bb7b8c57d4b6f274)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:45.951645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84077075-ae80-498b-900f-50be65457036 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:45.963978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] MFA Rules not processed for user `0ac69ac0663d425a9e1beeb3f48f8c46`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:45.965202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:45.994112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:46.012764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0f61dde2b24338bb7b8c57d4b6f274', 'request_id': 'req-84077075-ae80-498b-900f-50be65457036'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.023975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0f61dde2b24338bb7b8c57d4b6f274', 'request_id': 'req-84077075-ae80-498b-900f-50be65457036'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.039104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0f61dde2b24338bb7b8c57d4b6f274', 'request_id': 'req-84077075-ae80-498b-900f-50be65457036'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.064527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-84077075-ae80-498b-900f-50be65457036 admin admin] The token cache is being invalidate because project 9a0f61dde2b24338bb7b8c57d4b6f274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:46.065029 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.065925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.068216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1077/2145] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:45 2022] DELETE /identity/v3/projects/9a0f61dde2b24338bb7b8c57d4b6f274 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:46.070037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:46.072524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.082039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1069/2146] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:46.082039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:46.109128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069c2a1-f418-469d-81c8-08df4289ab25 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:46.151812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84077075-ae80-498b-900f-50be65457036 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:46.155606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:46.156645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:46.156645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:46.156846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:46.171298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.229575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:46.229575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:46.229787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:46.230826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] PATH_INFO: `/v3/users/c98dd1f925214fa5b7e71ea237ad572d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:46.235948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c98dd1f925214fa5b7e71ea237ad572d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:46.238407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:46.271578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:46.309074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:46.311310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:46.311697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c98dd1f925214fa5b7e71ea237ad572d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.322841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c98dd1f925214fa5b7e71ea237ad572d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.327145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1078/2147] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:46.328587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:46.330306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.354420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689a122-6da7-4d12-b1ba-aca5e15e6e50 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:46.365024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:46.366512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:46.366993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:46.367429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:46.369654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.386001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c98dd1f925214fa5b7e71ea237ad572d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.386988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c98dd1f925214fa5b7e71ea237ad572d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.407317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c98dd1f925214fa5b7e71ea237ad572d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:46.422706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1070/2148] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:46 2022] DELETE /identity/v3/users/c98dd1f925214fa5b7e71ea237ad572d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:46.531764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:46.533557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:46.557542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1079/2149] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:46.796452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:46.797892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.935913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c88be9-7696-467d-9703-423b173f87e8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:46.947769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:46.948500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:46.948754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:46.949027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:46.950846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.957239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:46.959873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:46.977996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:46.993597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675fbd1-fe56-4d7c-b201-0f08d822bb5d service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:47.128434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:47.148478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94ba64c-5dee-48e4-ac77-f9ef5bf707b0 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:47.163301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:47.163973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:47.164616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:47.164893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] PATH_INFO: `/v3/projects/81e8d52e5b674335b6546ddf5842024b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:47.166809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e8d52e5b674335b6546ddf5842024b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:47.172676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:47.179512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1071/2150] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:47.279039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e8d52e5b674335b6546ddf5842024b', 'request_id': 'req-429e37c8-dcf0-4342-a8d1-17704acb740a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.292169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e8d52e5b674335b6546ddf5842024b', 'request_id': 'req-429e37c8-dcf0-4342-a8d1-17704acb740a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.315887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e8d52e5b674335b6546ddf5842024b', 'request_id': 'req-429e37c8-dcf0-4342-a8d1-17704acb740a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.346644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] The token cache is being invalidate because project 81e8d52e5b674335b6546ddf5842024b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:47.347271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.352281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.355041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1080/2151] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:46 2022] DELETE /identity/v3/projects/81e8d52e5b674335b6546ddf5842024b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:47.362321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01fb78-f475-4379-892d-9e5a303ac53e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:47.362624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01fb78-f475-4379-892d-9e5a303ac53e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:47.367972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01fb78-f475-4379-892d-9e5a303ac53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:47.401045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eb01fb78-f475-4379-892d-9e5a303ac53e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:47.402393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb01fb78-f475-4379-892d-9e5a303ac53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:47.542950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:47.544361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:47.545482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:47.575730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1072/2152] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:46:47.585376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:47.585697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:47.586107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:47.645385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:47.646332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:47.661123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1073/2153] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:47.668356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:47.686057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f69d4-c0a9-4cab-bd65-1a1f983de103 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:47.695170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:47.695985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:47.696366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:47.696817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:47.697901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:47.700309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:47.705969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:47.748121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1074/2154] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:46:47.756908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:47.766742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e37c8-dcf0-4342-a8d1-17704acb740a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:47.773838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:47.774558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:47.774782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:47.775060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] PATH_INFO: `/v3/users/42bfc5626ee649d1bfc6472f87d94943` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:47.778242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42bfc5626ee649d1bfc6472f87d94943)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:47.778559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:47.781310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3021a9a4-53d9-475e-894b-3394fcf2f826 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:47.793574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:47.795091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:47.802112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:47.803522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:47.804786 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:47.807564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:47.851500 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:47.884648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1075/2155] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:47 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:47.889097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:47.896838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42bfc5626ee649d1bfc6472f87d94943'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.921797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42bfc5626ee649d1bfc6472f87d94943'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.944758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42bfc5626ee649d1bfc6472f87d94943'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.947496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42bfc5626ee649d1bfc6472f87d94943'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:47.995356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-26f20d44-81f1-4ccc-ab1c-be5e20df02a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42bfc5626ee649d1bfc6472f87d94943'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:48.025747 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1081/2156] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:47 2022] DELETE /identity/v3/users/42bfc5626ee649d1bfc6472f87d94943 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:48.029980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.030265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.030533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.062343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:48.063564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:48.066827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293779b9-201c-4444-a9b8-aa53c9108d7a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:48.182588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:48.186677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1082/2157] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:48.188461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.189658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.191566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.193009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:48.194401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:48.197348 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:48.197941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:48.240645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1076/2158] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:47 2022] POST /identity/v3/users => generated 291 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:48.245448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:48.261426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:48.289084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b7b45-d96b-4b91-b6f1-7d33b538c274 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:48.300479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:48.301114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.301541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.301645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.302483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:48.302981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b0d0c-f08d-4fea-bb16-b21bfe99447f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:48.304073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:48.307260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:48.307949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.308231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.308503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.316694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1077/2159] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:48.321266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:48.327676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:48.329638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:48.336653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc92fc-feb5-4cfb-b926-8c5857fe5a8c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:48.344396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1083/2160] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:48.346195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef002510-bf73-440c-9d32-0d564ac9e03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:48.346843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002510-bf73-440c-9d32-0d564ac9e03d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.347132 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002510-bf73-440c-9d32-0d564ac9e03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.347456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef002510-bf73-440c-9d32-0d564ac9e03d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.348148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002510-bf73-440c-9d32-0d564ac9e03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:48.350428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef002510-bf73-440c-9d32-0d564ac9e03d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:48.366974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1078/2161] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:48.374276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:48.388417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bacb5-d293-4ab8-960f-72f47ceb7434 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:48.399391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:48.401010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.401010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.401010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] PATH_INFO: `/v3/projects/615dacd34d544f6b97b571727897578e/users/3c48e5a70eab4f11a336bcc0bd89fdab/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.428723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615dacd34d544f6b97b571727897578e, user_id=3c48e5a70eab4f11a336bcc0bd89fdab, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:48.431929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:48.449630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1084/2162] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:48 2022] PUT /identity/v3/projects/615dacd34d544f6b97b571727897578e/users/3c48e5a70eab4f11a336bcc0bd89fdab/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:48.462046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.462517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.462628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.504868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] MFA Rules not processed for user `3c48e5a70eab4f11a336bcc0bd89fdab`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:48.506919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:48.523375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:48.646002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1079/2163] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:48.653704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:48.713380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:48.796267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037a1f-3128-4c0a-9914-720afb780f48 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:48.811110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:48.814870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:48.815378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:48.815796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:48.816734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:48.819107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:48.880444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1085/2164] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:48 2022] POST /identity/v3/projects => generated 381 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:48.882406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:48.906438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b7752-3648-460d-8419-a0da33dde91e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.023172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.023852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.024142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.024500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.025157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.029088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.051261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:49.054633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:49.087421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357d978-96d1-4949-8b3b-96c34906a9a9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.092705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.096900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.096900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.096900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] PATH_INFO: `/v3/projects/cb1ea233e49e4827a3848c613a2b9d46` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.097712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1ea233e49e4827a3848c613a2b9d46)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.098035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1080/2165] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:48 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:49.099227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:49.100495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.128008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b46a8c-348f-4b57-958c-20f73cb8747f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.138814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.139936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.140156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.142766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.144308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.150878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.186714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1081/2166] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:49.190333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:49.231390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff3574-b029-48b2-bfed-b4551d361a10 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.236137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1ea233e49e4827a3848c613a2b9d46', 'request_id': 'req-cbc952b7-1875-4a47-be15-084c66f4880c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.256674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1ea233e49e4827a3848c613a2b9d46', 'request_id': 'req-cbc952b7-1875-4a47-be15-084c66f4880c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.267240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1ea233e49e4827a3848c613a2b9d46', 'request_id': 'req-cbc952b7-1875-4a47-be15-084c66f4880c'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.286386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] The token cache is being invalidate because project cb1ea233e49e4827a3848c613a2b9d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:49.287768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.292524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.296829 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1086/2167] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:49 2022] DELETE /identity/v3/projects/cb1ea233e49e4827a3848c613a2b9d46 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:49.301039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:49.304678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:49.389528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.390210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.390482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.390755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] PATH_INFO: `/v3/projects/a4e2990242294523b98554225777706f/users/9b868f2116ff4c8d8470c8b78915106e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.411130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc952b7-1875-4a47-be15-084c66f4880c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.413175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e2990242294523b98554225777706f, user_id=9b868f2116ff4c8d8470c8b78915106e, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.424705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.425304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.425605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.427118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] PATH_INFO: `/v3/users/0ac69ac0663d425a9e1beeb3f48f8c46` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.429461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ac69ac0663d425a9e1beeb3f48f8c46)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.432810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.438141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.472694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1082/2168] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:49 2022] PUT /identity/v3/projects/a4e2990242294523b98554225777706f/users/9b868f2116ff4c8d8470c8b78915106e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:49.478356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:49.482619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:49.534751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:49.541821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:49.552369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ac69ac0663d425a9e1beeb3f48f8c46'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.568690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ac69ac0663d425a9e1beeb3f48f8c46'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.592738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ac69ac0663d425a9e1beeb3f48f8c46'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.594310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ac69ac0663d425a9e1beeb3f48f8c46'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.638486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dffa1a-3499-41fc-865c-8f1c5a75b874 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.646182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ac69ac0663d425a9e1beeb3f48f8c46'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:49.647479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc344200-587b-493a-8a36-29c83e26b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.648173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344200-587b-493a-8a36-29c83e26b388 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.648456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344200-587b-493a-8a36-29c83e26b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.648728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc344200-587b-493a-8a36-29c83e26b388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.649648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc344200-587b-493a-8a36-29c83e26b388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.654402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc344200-587b-493a-8a36-29c83e26b388 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.667119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1087/2169] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:49 2022] DELETE /identity/v3/users/0ac69ac0663d425a9e1beeb3f48f8c46 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:49.687922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1083/2170] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:49.689677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:49.691512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:49.890947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5918ad6-98d4-4588-9a44-276461a4fbb9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:49.903278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:49.904065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.904384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.904713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] PATH_INFO: `/v3/projects/a4e2990242294523b98554225777706f/users/9b868f2116ff4c8d8470c8b78915106e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:49.912330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e2990242294523b98554225777706f, user_id=9b868f2116ff4c8d8470c8b78915106e, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:49.915033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:49.954612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1088/2171] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:49 2022] PUT /identity/v3/projects/a4e2990242294523b98554225777706f/users/9b868f2116ff4c8d8470c8b78915106e/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:49.956027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:49.956337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:49.956707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.050594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] MFA Rules not processed for user `9b868f2116ff4c8d8470c8b78915106e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:50.052101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.239177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1084/2172] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:50.246125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:50.249040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.256973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:50.258410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.351291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abe62-d276-4fc5-ba24-f03745673280 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:50.356991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:50.357790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.358137 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.358488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] PATH_INFO: `/v3/projects/f20070b13b534d73b0977b50686fcc3a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.360639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20070b13b534d73b0977b50686fcc3a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:50.362623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:50.362797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ba58c-823d-4773-9f35-cde519c0e767 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:50.368713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e91c89-c960-47fb-b338-8926146735f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:50.369435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91c89-c960-47fb-b338-8926146735f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.369849 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91c89-c960-47fb-b338-8926146735f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.370311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91c89-c960-47fb-b338-8926146735f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.392524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e91c89-c960-47fb-b338-8926146735f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:50.394652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e91c89-c960-47fb-b338-8926146735f7 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:50.424199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1085/2173] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:50.425149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20070b13b534d73b0977b50686fcc3a', 'request_id': 'req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:50.430993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:50.434660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20070b13b534d73b0977b50686fcc3a', 'request_id': 'req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:50.445726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20070b13b534d73b0977b50686fcc3a', 'request_id': 'req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:50.465278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] The token cache is being invalidate because project f20070b13b534d73b0977b50686fcc3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:50.465816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:50.466601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:50.472070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1089/2174] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:50 2022] DELETE /identity/v3/projects/f20070b13b534d73b0977b50686fcc3a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:50.473781 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b84fc6-881b-4e79-889f-16be16880c7c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.474305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b84fc6-881b-4e79-889f-16be16880c7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.474473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b84fc6-881b-4e79-889f-16be16880c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.501108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d8b84fc6-881b-4e79-889f-16be16880c7c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:50.502507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d8b84fc6-881b-4e79-889f-16be16880c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.534593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:50.536219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.591413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:50.671109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d3ddf-bc4c-41c3-864e-1901a0ce1dcb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:50.675195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1086/2175] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:50.686339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.686705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.687037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.692296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:50.692925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.693444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.693869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.697238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.738345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:50.740981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:50.758255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1087/2176] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:50.763640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:50.788509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f400b87-14dd-4b0b-a23f-96ac83e4aace None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:50.804519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:50.805208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.805542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.805750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.806427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:50.806721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:50.811326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:50.841623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1088/2177] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:50.845565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:50.845965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:50.848545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:50.873423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1090/2178] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:50.935763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882cd5b6-d616-49c9-b4d5-cdd3df3e362f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:50.957220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:50.959841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:50.961973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:50.962777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:50.963613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:50.966416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:51.066192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1089/2179] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:50 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:51.076202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:51.090331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6425a-b40b-4365-b33e-e6f095fd4271 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:51.172315 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:51.172935 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:51.173232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:51.173470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:51.174114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:51.176089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:51.222290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1091/2180] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:51 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:51.227433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:51.244661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484526a-3e3b-4e46-b605-6ef12796ba17 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:51.268249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:51.269516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:51.269807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:51.270080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:51.270746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:51.272973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:51.303370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1090/2181] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:51.312896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:51.326013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e96c10-be9b-43a1-8060-95dc9062c5b7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:51.331478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8e3b0-f1b9-44cc-bfa6-56498d558ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:51.332613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8e3b0-f1b9-44cc-bfa6-56498d558ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:51.333043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8e3b0-f1b9-44cc-bfa6-56498d558ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:51.333824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8e3b0-f1b9-44cc-bfa6-56498d558ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:51.334861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8e3b0-f1b9-44cc-bfa6-56498d558ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:51.337135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8e3b0-f1b9-44cc-bfa6-56498d558ca2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:51.375962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1092/2182] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:51 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:51.377465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:51.396103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc342-6f78-4b91-80c7-19bcff29f516 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:51.403709 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:51.404456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:51.404698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:51.404998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] PATH_INFO: `/v3/projects/c282713d51e74b6b87a7c0024cc18eb8/users/aa8ebc429a4a45ae8742252218799f3b/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:51.425028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c282713d51e74b6b87a7c0024cc18eb8, user_id=aa8ebc429a4a45ae8742252218799f3b, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:51.428603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:51.450499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1091/2183] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:51 2022] PUT /identity/v3/projects/c282713d51e74b6b87a7c0024cc18eb8/users/aa8ebc429a4a45ae8742252218799f3b/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:51.463234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:51.463520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:51.463833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:51.523051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] MFA Rules not processed for user `aa8ebc429a4a45ae8742252218799f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:51.524708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:51.530792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:51.677596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1093/2184] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:51.694904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:51.699125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:51.852428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e44bd-8588-4f68-b50d-0573ed03bfdc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:51.859288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2010a08-421c-439d-a2d3-0261d3a54fa8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:51.860326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2010a08-421c-439d-a2d3-0261d3a54fa8 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:51.860747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2010a08-421c-439d-a2d3-0261d3a54fa8 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:51.861112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2010a08-421c-439d-a2d3-0261d3a54fa8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:51.875988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2010a08-421c-439d-a2d3-0261d3a54fa8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:51.882279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2010a08-421c-439d-a2d3-0261d3a54fa8 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:51.906864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1092/2185] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:51.917689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:51.921423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:52.112356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cb4c2-0d3d-444f-9eb3-bd1f2ac8105c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:52.125714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:52.125714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:52.125714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:52.125714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] PATH_INFO: `/v3/users/3c48e5a70eab4f11a336bcc0bd89fdab` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:52.127336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c48e5a70eab4f11a336bcc0bd89fdab)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:52.140810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:52.274963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c48e5a70eab4f11a336bcc0bd89fdab'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:52.291765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c48e5a70eab4f11a336bcc0bd89fdab'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:52.306045 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c48e5a70eab4f11a336bcc0bd89fdab'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:52.307052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c48e5a70eab4f11a336bcc0bd89fdab'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:52.326818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c48e5a70eab4f11a336bcc0bd89fdab'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:52.334183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1094/2186] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:51 2022] DELETE /identity/v3/users/3c48e5a70eab4f11a336bcc0bd89fdab => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:52.339820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81eef-9e50-4275-87b4-dbd29a453284 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:52.340318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81eef-9e50-4275-87b4-dbd29a453284 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:52.341539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b81eef-9e50-4275-87b4-dbd29a453284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:52.377735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-62b81eef-9e50-4275-87b4-dbd29a453284 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:52.379780 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-62b81eef-9e50-4275-87b4-dbd29a453284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:52.492907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:52.519844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1093/2187] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:52.531128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:52.532616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:52.579903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae9c31-241d-426a-a0a6-38b4040b9245 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:52.580238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae9c31-241d-426a-a0a6-38b4040b9245 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:52.580514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae9c31-241d-426a-a0a6-38b4040b9245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:52.631973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:52.648268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-44ae9c31-241d-426a-a0a6-38b4040b9245 None None] MFA Rules not processed for user `cfe6eb38e8df492daaef5755ddea7773`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:52.651080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44ae9c31-241d-426a-a0a6-38b4040b9245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:52.703518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8da86-5ad7-4296-9bc1-1492a79cb7bc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:52.727302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:52.731242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:52.731729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:52.734046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:52.757577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:52.759116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:52.782805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1094/2188] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:52.796334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:52.796621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:52.796895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:52.802645 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1095/2189] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:52.850228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] MFA Rules not processed for user `cfe6eb38e8df492daaef5755ddea7773`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:52.851475 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:52.862590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1095/2190] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:52.873241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:52.874989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:53.045437 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8e066-ab4a-459d-afc6-2d270ca585dd service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:53.052414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:53.054332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:53.054332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:53.054332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:53.063465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:53.065380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:53.076245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1096/2191] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:53.265301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:53.266720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:53.331869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa20104-1b1b-4cb8-8c12-4c045a880a6d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:53.367296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:53.368010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:53.368435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:53.368758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] PATH_INFO: `/v3/projects/615dacd34d544f6b97b571727897578e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:53.370036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615dacd34d544f6b97b571727897578e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:53.373797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:53.449674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615dacd34d544f6b97b571727897578e', 'request_id': 'req-383e5a3b-ef75-4036-8813-48c2273024d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.460143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615dacd34d544f6b97b571727897578e', 'request_id': 'req-383e5a3b-ef75-4036-8813-48c2273024d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.470291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615dacd34d544f6b97b571727897578e', 'request_id': 'req-383e5a3b-ef75-4036-8813-48c2273024d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.486978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] The token cache is being invalidate because project 615dacd34d544f6b97b571727897578e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:53.487856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.501158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-383e5a3b-ef75-4036-8813-48c2273024d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.504869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1096/2192] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:53 2022] DELETE /identity/v3/projects/615dacd34d544f6b97b571727897578e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:53.511162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:53.531333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21438562-c6ca-4b35-910c-b2be5402deb9 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:53.624878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:53.627518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:53.627933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:53.628223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] PATH_INFO: `/v3/users/9b868f2116ff4c8d8470c8b78915106e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:53.630485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b868f2116ff4c8d8470c8b78915106e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:53.632578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:53.694817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b868f2116ff4c8d8470c8b78915106e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.705311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b868f2116ff4c8d8470c8b78915106e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.707579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:53.715228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b868f2116ff4c8d8470c8b78915106e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.716294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b868f2116ff4c8d8470c8b78915106e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.740261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b868f2116ff4c8d8470c8b78915106e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:53.748285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1097/2193] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:53 2022] DELETE /identity/v3/users/9b868f2116ff4c8d8470c8b78915106e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:54.077449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:54.077904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:54.077980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:54.111259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] MFA Rules not processed for user `cfe6eb38e8df492daaef5755ddea7773`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:54.115049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.203523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:54.231706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1097/2194] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:54.302570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:54.303610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:54.306410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.306961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.424239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fc63f-ea48-4ba7-8c15-e05bb3d9cd43 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:54.431882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:54.432746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:54.433055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:54.433306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] PATH_INFO: `/v3/users/ecd52acf2a3747f498635880604c8219` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:54.434830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd52acf2a3747f498635880604c8219)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:54.437980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:54.464284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c9538-59ae-4ff1-a188-ba46e4e5a1f0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:54.475365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:54.476710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:54.482551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:54.483051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] PATH_INFO: `/v3/projects/a4e2990242294523b98554225777706f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:54.485886 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e2990242294523b98554225777706f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:54.507057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:54.526813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd52acf2a3747f498635880604c8219'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.559111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd52acf2a3747f498635880604c8219'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.578114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd52acf2a3747f498635880604c8219'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.579021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd52acf2a3747f498635880604c8219'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.635521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b168b875-d64d-4607-9363-306fd75c7bdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd52acf2a3747f498635880604c8219'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.648911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1098/2195] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:54 2022] DELETE /identity/v3/users/ecd52acf2a3747f498635880604c8219 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:54.654867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:54.655622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:54.656182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:54.674307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e2990242294523b98554225777706f', 'request_id': 'req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.688290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e2990242294523b98554225777706f', 'request_id': 'req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.701181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:54.703980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.705161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e2990242294523b98554225777706f', 'request_id': 'req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.732882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] The token cache is being invalidate because project a4e2990242294523b98554225777706f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:54.733636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.735114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7e75919-3bc8-4896-89e6-0ea6cac90eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:54.737391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1098/2196] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:54 2022] DELETE /identity/v3/projects/a4e2990242294523b98554225777706f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:54.747407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b429-750c-44e8-9d29-323f91482ccf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:54.747596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b429-750c-44e8-9d29-323f91482ccf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:54.747783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0b429-750c-44e8-9d29-323f91482ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:54.787728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4ac0b429-750c-44e8-9d29-323f91482ccf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:54.789217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ac0b429-750c-44e8-9d29-323f91482ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.815213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:54.858064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1099/2197] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:54.869449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:54.873632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.905277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1099/2198] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:54.917076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:54.917386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:54.917876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:54.951085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:54.953356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:54.971938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1100/2199] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:54.979092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.000991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4530-1722-4789-8276-b66fa7eae54f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.009708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.010345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.010623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.010896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.011622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:55.011983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.014109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.026293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c47b0-ef89-438f-994a-e4c71ee299bb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.028983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1101/2200] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:46:55.033448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.033827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.034181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.034593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.035042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.052226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.052503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d3333-15f3-4b96-924c-2f4508520048 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.054440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.057620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.058284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.058727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.059625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.059625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.061790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.088907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:55.101776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1100/2201] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:55.116614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1102/2202] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:55 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:55.134022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.150933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7994d42-c474-4d64-8edb-e681d2bbc15e service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.254142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.254766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.255042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.255492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.257408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.259655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.301047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1101/2203] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:55 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:55.305517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.320580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0b8d-41de-4fef-8863-ecf47a5c0012 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.331965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.332666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.333000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.333249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.334179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.337783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.350692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1103/2204] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:55.355788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.367387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8544e-9793-42fd-9504-6f065a9e0d48 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.372463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668f346-83b4-4b44-9bf7-c8a6c560672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.373094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668f346-83b4-4b44-9bf7-c8a6c560672c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.373372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668f346-83b4-4b44-9bf7-c8a6c560672c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.373638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668f346-83b4-4b44-9bf7-c8a6c560672c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.374299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668f346-83b4-4b44-9bf7-c8a6c560672c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.375984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668f346-83b4-4b44-9bf7-c8a6c560672c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.391605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1102/2205] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:55.392788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.407666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19096051-d7fc-4a3f-83da-b39fe3a7cac2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.413198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.413891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.414168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.414438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] PATH_INFO: `/v3/projects/8997133d28b04001bbc7861ab9e1fae4/users/87bd4be92bf7497b99826bd406e3644b/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.449618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8997133d28b04001bbc7861ab9e1fae4, user_id=87bd4be92bf7497b99826bd406e3644b, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.452887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.470195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1104/2206] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:55 2022] PUT /identity/v3/projects/8997133d28b04001bbc7861ab9e1fae4/users/87bd4be92bf7497b99826bd406e3644b/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:55.484853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.485129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.485426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.511364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.513990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:55.549136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] MFA Rules not processed for user `87bd4be92bf7497b99826bd406e3644b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:55.550725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:55.609858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fffe3-055d-461b-ad24-9810cd7d0b29 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.620041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.624928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.625331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.625747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] PATH_INFO: `/v3/projects/110eba3edd4b493fb4c1cd83a32986c5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.636319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] RBAC: Authorizing `identity:delete_project(project_id=110eba3edd4b493fb4c1cd83a32986c5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.639100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.645582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1103/2207] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:55.652852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.660793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:55.673695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59dc78-8b9a-4b2a-94a2-061cc61363e9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.679580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.680270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.680547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.680825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.681555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.683590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.732560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1104/2208] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:55 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:55.741511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.741858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '110eba3edd4b493fb4c1cd83a32986c5', 'request_id': 'req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:55.770392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '110eba3edd4b493fb4c1cd83a32986c5', 'request_id': 'req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:55.789610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '110eba3edd4b493fb4c1cd83a32986c5', 'request_id': 'req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:55.827326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] The token cache is being invalidate because project 110eba3edd4b493fb4c1cd83a32986c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:55.827577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:55.828494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:55.831795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1105/2209] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:55 2022] DELETE /identity/v3/projects/110eba3edd4b493fb4c1cd83a32986c5 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:55.836602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.839206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:55.898507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192f11b-fd11-4085-82ef-802a5e0bc7e2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.907446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.909951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.910556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.911166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.914238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.918749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:55.919195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.955428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c6b8a-4361-43c7-95f4-5e63eb0dd43b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:55.960977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:55.962068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:55.962338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:55.962616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] PATH_INFO: `/v3/users/2872d2988f4f49b1b2f7d208769998f0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:55.963969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=2872d2988f4f49b1b2f7d208769998f0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:55.966307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:55.986252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1105/2210] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:55 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:55.991024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:55.994284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:56.026507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db996e-be44-4bc2-a860-78e2d9503fdf admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:56.036419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:56.037599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:56.037880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:56.038421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:56.039207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:56.041970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:56.059284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1106/2211] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:56.060380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:56.071171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2872d2988f4f49b1b2f7d208769998f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.072882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa112466-9ec8-4a53-ada1-970d13ac0530 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:56.082706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:56.083310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:56.083593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:56.083993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:56.084714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:56.085470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2872d2988f4f49b1b2f7d208769998f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.086585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:56.098757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1107/2212] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:56.104261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:56.120394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fb610-dc43-433d-b790-11f764dc7d24 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:56.121215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2872d2988f4f49b1b2f7d208769998f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.122107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2872d2988f4f49b1b2f7d208769998f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.137739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-44a94e08-07e5-4676-86b5-e17faa03c7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2872d2988f4f49b1b2f7d208769998f0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.145098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1106/2213] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:46:55 2022] DELETE /identity/v3/users/2872d2988f4f49b1b2f7d208769998f0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:56.159884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:56.163813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:56.164339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:56.164822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] PATH_INFO: `/v3/projects/ed267396a0ff4ce4a88ebd72ea5577b6/users/f367338eb78a4f79b08384cd5baaf8d3/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:56.255919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed267396a0ff4ce4a88ebd72ea5577b6, user_id=f367338eb78a4f79b08384cd5baaf8d3, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:56.274887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:56.327214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1108/2214] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:56 2022] PUT /identity/v3/projects/ed267396a0ff4ce4a88ebd72ea5577b6/users/f367338eb78a4f79b08384cd5baaf8d3/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:56.339003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:56.339294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:56.339617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:56.404400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] MFA Rules not processed for user `f367338eb78a4f79b08384cd5baaf8d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:56.405789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:56.529698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1107/2215] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:56.535163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:56.538017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:56.739499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675cd14f-e1ba-4e19-bbd7-c92d0f9fe9da admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:56.749374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:56.750014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:56.750291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:56.750548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:56.751213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:56.754662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:56.766920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:56.767306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:56.825218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e071e-603d-4056-91b6-f7f8b6f1c9d5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:56.843339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:56.844735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:56.846546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:56.848851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] PATH_INFO: `/v3/projects/3b23b74cc4a44c389008f2961ad9a3d9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:56.852839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b23b74cc4a44c389008f2961ad9a3d9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:56.862746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1109/2216] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:56 2022] POST /identity/v3/projects => generated 375 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:46:56.862845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:56.863580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:56.887386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:56.977288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:56.979230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b23b74cc4a44c389008f2961ad9a3d9', 'request_id': 'req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.991461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b23b74cc4a44c389008f2961ad9a3d9', 'request_id': 'req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:56.999235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400a642-09ef-47ae-b446-5a910bda3b8b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.002317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b23b74cc4a44c389008f2961ad9a3d9', 'request_id': 'req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:57.020650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] The token cache is being invalidate because project 3b23b74cc4a44c389008f2961ad9a3d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:46:57.020983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:57.021785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7b9a2aea-dc2e-42f9-9d30-bd44a52118d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:46:57.023320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1108/2217] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:46:56 2022] DELETE /identity/v3/projects/3b23b74cc4a44c389008f2961ad9a3d9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:57.028630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7ed7d-7a20-417a-a169-21c1448c4702 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.028982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7ed7d-7a20-417a-a169-21c1448c4702 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.029232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7ed7d-7a20-417a-a169-21c1448c4702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.051496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d7a7ed7d-7a20-417a-a169-21c1448c4702 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:57.053364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d7a7ed7d-7a20-417a-a169-21c1448c4702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:57.085678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.086559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.087456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.087826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.088555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.091103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.123802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1110/2218] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:56 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:57.128948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.139392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:57.160584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5675bca-5dfc-49b0-8ba8-b91b0103fb64 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.164539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1109/2219] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:57.167050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.167394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.167772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.170846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.171907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.172194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.176604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.178864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.180092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.202313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:57.204961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:57.208456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1111/2220] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:46:57.213071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.215291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1110/2221] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:57.223480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.227155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f9b14-841b-4a1e-bde9-2943506baa78 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.233780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.234983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.235131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512f0f9-52df-4fbb-98a3-2b1a87065e6e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.235458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.235865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] PATH_INFO: `/v3/projects/2d1e93c98fcf43f4b4e869dd9a4c058d/users/53e7456d8e3449b889ff1de6d91ff45c/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.242795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.243850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.243850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.244103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.244778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:46:57.245083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.246851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.255032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1e93c98fcf43f4b4e869dd9a4c058d, user_id=53e7456d8e3449b889ff1de6d91ff45c, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.258704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.279286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1112/2222] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:57 2022] PUT /identity/v3/projects/2d1e93c98fcf43f4b4e869dd9a4c058d/users/53e7456d8e3449b889ff1de6d91ff45c/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:57.283854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1111/2223] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:46:57.285729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.287602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.306157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcafc5-aa23-4007-be94-f4d5a951b13d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.433042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee0c424-ad44-4e2b-8d48-9e3a5dc6cddd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.441422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.455243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.455243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.455243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.455706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.455706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.455706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.455706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.456020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.456103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.456174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.456243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.463476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1113/2224] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:57.467468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.493763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c378729-50ce-4755-9dc0-b64e139c5f9f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.512413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1112/2225] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:46:57 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:57.513469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.517162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9cc0a-0190-4247-b586-5964290ebccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.517856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9cc0a-0190-4247-b586-5964290ebccf admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.518135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9cc0a-0190-4247-b586-5964290ebccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.518410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9cc0a-0190-4247-b586-5964290ebccf admin admin] PATH_INFO: `/v3/projects/2d1e93c98fcf43f4b4e869dd9a4c058d/users/53e7456d8e3449b889ff1de6d91ff45c/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.529636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9cc0a-0190-4247-b586-5964290ebccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1e93c98fcf43f4b4e869dd9a4c058d, user_id=53e7456d8e3449b889ff1de6d91ff45c, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.532338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9cc0a-0190-4247-b586-5964290ebccf admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.534897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:57.545792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1114/2226] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:57 2022] PUT /identity/v3/projects/2d1e93c98fcf43f4b4e869dd9a4c058d/users/53e7456d8e3449b889ff1de6d91ff45c/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:46:57.557151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.557513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.557860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.592118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:57.605876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] MFA Rules not processed for user `53e7456d8e3449b889ff1de6d91ff45c`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:57.608018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:57.620239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aee6db-56da-4b5c-82ca-2ff1f456ef37 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.719061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.719722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.721663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.721663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.722930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.727447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.741333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1115/2227] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:57.753843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.755726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:57.773517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1113/2228] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:46:57 2022] POST /identity/v3/users => generated 292 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:46:57.774911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.787890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c63a03-5216-4b98-8289-c7140ea70c22 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.796937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.798108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.798503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.798814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.802203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.802203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.818447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1114/2229] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:57.820065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.863770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186e24c-d36d-43ef-9184-5cfa6830c37e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.864002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f89b95-a9d4-43b4-9374-00b61ce79d58 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.864089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.864089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.864089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.864089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.864373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.864373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.869547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.869547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.869547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.869547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.874337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:57.890218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1115/2230] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:46:57.897522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:57.925103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e69f0c-eb51-4592-a814-f799f0ad8b4c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:57.940735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:57.941495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:57.941815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:57.942153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] PATH_INFO: `/v3/projects/56f0f0359108449dacfa08a2cf4de4f4/users/bf8613b79aa24e818502bd5bbb0e62b4/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:57.968048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f0f0359108449dacfa08a2cf4de4f4, user_id=bf8613b79aa24e818502bd5bbb0e62b4, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:57.976693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:57.986738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:58.007498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2777d8-9bb7-4496-8ed4-1f4a5b584fe9 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:58.026522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1116/2232] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:46:58.035727 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1116/2232] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:46:57 2022] PUT /identity/v3/projects/56f0f0359108449dacfa08a2cf4de4f4/users/bf8613b79aa24e818502bd5bbb0e62b4/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:46:58.047890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:58.050967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:58.050967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:58.112126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] MFA Rules not processed for user `bf8613b79aa24e818502bd5bbb0e62b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:46:58.113768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:58.167867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:46:58.273902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1117/2233] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:46:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:46:58.336412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:46:58.342274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:46:58.460303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e6d62-15c3-4647-b478-3f473be4bf4b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:46:58.465873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:46:58.466592 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:46:58.466902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:46:58.467268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:46:58.482057 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:46:58.484744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:46:58.528666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1117/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:46:59.993295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:00.002949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061211f-80aa-4c61-90c4-b95dbaf92139 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:00.007102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:00.007699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:00.008001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:00.008266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:00.010006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:00.070653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:00.072172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:00.081238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1118/2235] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:46:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:00.854767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:00.863893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ebdc-9f17-493a-9db1-90f5049092f0 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:00.868636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:00.869559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:00.870302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:00.870618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:00.908035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:00.940945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:00.942710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:00.953208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1118/2236] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:06.101236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.212804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c6ee5-af79-4f10-8b62-00c6b04aa086 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:06.218961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:06.220390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.220893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.221370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] PATH_INFO: `/v3/users/87bd4be92bf7497b99826bd406e3644b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.222761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87bd4be92bf7497b99826bd406e3644b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:06.225158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:06.290451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.293742 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.302523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87bd4be92bf7497b99826bd406e3644b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.314016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87bd4be92bf7497b99826bd406e3644b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.326060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ce78af-1f35-45d1-ad2e-6572bd77d8fa service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:06.328612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87bd4be92bf7497b99826bd406e3644b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.329865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87bd4be92bf7497b99826bd406e3644b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.331622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:06.332491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:06.334266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.334955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.335455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] PATH_INFO: `/v3/users/30363c264f9b4ff5b5e17654bc3e9000` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.337110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30363c264f9b4ff5b5e17654bc3e9000)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:06.340008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:06.358625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-27c1db57-1f94-4160-baaa-98f6b011140a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87bd4be92bf7497b99826bd406e3644b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.371602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1119/2237] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:06 2022] DELETE /identity/v3/users/87bd4be92bf7497b99826bd406e3644b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:06.376272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.377440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.377715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.415100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:06.416637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.432000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30363c264f9b4ff5b5e17654bc3e9000'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.447151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30363c264f9b4ff5b5e17654bc3e9000'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.472404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30363c264f9b4ff5b5e17654bc3e9000'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.476677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30363c264f9b4ff5b5e17654bc3e9000'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.487462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30363c264f9b4ff5b5e17654bc3e9000'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.496194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1119/2238] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:06 2022] DELETE /identity/v3/users/30363c264f9b4ff5b5e17654bc3e9000 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:06.524393 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.525995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.535070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1120/2239] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:06.556425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.558678 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.622599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256b133-8328-4cd3-8059-7b6e6e2f7417 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:06.630690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:06.631993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.632608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.632910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] PATH_INFO: `/v3/users/bf8613b79aa24e818502bd5bbb0e62b4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.635777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8613b79aa24e818502bd5bbb0e62b4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:06.638524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:06.647127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3485b42-e3c7-4cc6-984e-8f46c8ae3414 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:06.699977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:06.702909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8613b79aa24e818502bd5bbb0e62b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.713430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8613b79aa24e818502bd5bbb0e62b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.720842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:06.721626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.721924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.722163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.733557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:06.734982 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:06.739244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8613b79aa24e818502bd5bbb0e62b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.740143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8613b79aa24e818502bd5bbb0e62b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.752699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5ad978c9-9415-44d4-9fb9-6152fd7f4d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8613b79aa24e818502bd5bbb0e62b4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.754475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1121/2240] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:06.755604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.756942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.761335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1120/2241] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:06 2022] DELETE /identity/v3/users/bf8613b79aa24e818502bd5bbb0e62b4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:06.769084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.769439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.769674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.773225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e3b8b-f86e-4d49-887c-70462fd622c4 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:06.794611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:06.796970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.824720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:06.850622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:06.851275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:06.851550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:06.852320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] PATH_INFO: `/v3/projects/d00b4ea7657d41e6a02531c3509619c9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:06.853688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00b4ea7657d41e6a02531c3509619c9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:06.856987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:06.887360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1121/2242] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:47:06.898244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.900212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:06.938640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00b4ea7657d41e6a02531c3509619c9', 'request_id': 'req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.949631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00b4ea7657d41e6a02531c3509619c9', 'request_id': 'req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.960414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00b4ea7657d41e6a02531c3509619c9', 'request_id': 'req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.981210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] The token cache is being invalidate because project d00b4ea7657d41e6a02531c3509619c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:06.981436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.982712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:06.984514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1122/2243] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:06 2022] DELETE /identity/v3/projects/d00b4ea7657d41e6a02531c3509619c9 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:06.986901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60157c-6d3b-48d9-8c7c-6798edc72c1e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:06.988347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:06.990160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:07.074734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:07.075529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.075878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.076148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.078180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:07.087665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3550b-cfae-407a-ba9c-2bf953d2eef7 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:07.095953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:07.097422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:07.100887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:07.101527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.102096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.102096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] PATH_INFO: `/v3/users/5be6d6edaabf476ca5189b11f44f275e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.105612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5be6d6edaabf476ca5189b11f44f275e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:07.107501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:07.112547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1122/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:47:07.123899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:07.129571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:07.147418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59e9ab-f58c-42bb-b251-ccc854e5e1f5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:07.154895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:07.156034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.156531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.156992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] PATH_INFO: `/v3/projects/8997133d28b04001bbc7861ab9e1fae4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.161028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8997133d28b04001bbc7861ab9e1fae4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:07.164920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:07.186526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:07.210565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5be6d6edaabf476ca5189b11f44f275e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.224208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5be6d6edaabf476ca5189b11f44f275e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.234178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5be6d6edaabf476ca5189b11f44f275e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.235958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5be6d6edaabf476ca5189b11f44f275e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.252677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:07.258625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8997133d28b04001bbc7861ab9e1fae4', 'request_id': 'req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.264101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5be6d6edaabf476ca5189b11f44f275e'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.272646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8997133d28b04001bbc7861ab9e1fae4', 'request_id': 'req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.272856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1123/2245] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:06 2022] DELETE /identity/v3/users/5be6d6edaabf476ca5189b11f44f275e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:07.290537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8997133d28b04001bbc7861ab9e1fae4', 'request_id': 'req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.306676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] The token cache is being invalidate because project 8997133d28b04001bbc7861ab9e1fae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:07.307074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.307793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.309358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1123/2246] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:07 2022] DELETE /identity/v3/projects/8997133d28b04001bbc7861ab9e1fae4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:07.313992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:07.315231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:07.458327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8b380-c12f-4c8c-b9b8-8eba4acd5ef0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:07.466691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:07.467576 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.468740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.469017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] PATH_INFO: `/v3/users/f367338eb78a4f79b08384cd5baaf8d3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.470856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f367338eb78a4f79b08384cd5baaf8d3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:07.473320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:07.519687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:07.523443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:07.560150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f367338eb78a4f79b08384cd5baaf8d3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.572808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f367338eb78a4f79b08384cd5baaf8d3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.598716 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f367338eb78a4f79b08384cd5baaf8d3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.603972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f367338eb78a4f79b08384cd5baaf8d3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.628061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f367338eb78a4f79b08384cd5baaf8d3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.636094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1124/2247] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:07 2022] DELETE /identity/v3/users/f367338eb78a4f79b08384cd5baaf8d3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:47:07.636857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:07.638562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:07.704189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cdaf2d-1b8f-4d06-8fe9-26e3c93b84cd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:07.801446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23b007-91bb-4617-9175-427bbae18a33 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:07.809387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:07.810159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.810456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.810880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] PATH_INFO: `/v3/projects/56f0f0359108449dacfa08a2cf4de4f4` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.812544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f0f0359108449dacfa08a2cf4de4f4)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:07.814771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:07.830073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:07.830733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:07.831635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.832985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.833604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] PATH_INFO: `/v3/projects/9ca129e74fe049d5a345febe9321d2e8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.835833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca129e74fe049d5a345febe9321d2e8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:07.838143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:07.908634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca129e74fe049d5a345febe9321d2e8', 'request_id': 'req-25b9117d-988a-4601-afb9-147735fdcc81'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.920367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:07.933004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca129e74fe049d5a345febe9321d2e8', 'request_id': 'req-25b9117d-988a-4601-afb9-147735fdcc81'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.941950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca129e74fe049d5a345febe9321d2e8', 'request_id': 'req-25b9117d-988a-4601-afb9-147735fdcc81'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.944171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f0f0359108449dacfa08a2cf4de4f4', 'request_id': 'req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.964998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] The token cache is being invalidate because project 9ca129e74fe049d5a345febe9321d2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:07.965480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.966284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25b9117d-988a-4601-afb9-147735fdcc81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.968630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1124/2248] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:07 2022] DELETE /identity/v3/projects/9ca129e74fe049d5a345febe9321d2e8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:07.977720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f0f0359108449dacfa08a2cf4de4f4', 'request_id': 'req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:07.978655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4df4-c792-4209-a72b-9062e997c028 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:07.979484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4df4-c792-4209-a72b-9062e997c028 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:07.980053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4df4-c792-4209-a72b-9062e997c028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:07.986464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f0f0359108449dacfa08a2cf4de4f4', 'request_id': 'req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.005233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] The token cache is being invalidate because project 56f0f0359108449dacfa08a2cf4de4f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:08.005533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.008854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2f1c4df4-c792-4209-a72b-9062e997c028 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:08.009004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-20f20ec0-1c1a-4ad6-a6bc-54a2fa8a8e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.010105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f1c4df4-c792-4209-a72b-9062e997c028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.013086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1125/2249] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:07 2022] DELETE /identity/v3/projects/56f0f0359108449dacfa08a2cf4de4f4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:08.026803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.027108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.027358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.051066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:08.052595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.094457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1125/2250] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:47:08.102672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2780-cda2-4bb7-afc3-55d8c3fbceda None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.102976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2780-cda2-4bb7-afc3-55d8c3fbceda None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.103235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb2780-cda2-4bb7-afc3-55d8c3fbceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.139730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f7bb2780-cda2-4bb7-afc3-55d8c3fbceda None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:08.141081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f7bb2780-cda2-4bb7-afc3-55d8c3fbceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.159792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1126/2251] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:08.160998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1126/2252] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:08.165926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.167165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.167643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.168183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.189096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66c615-4705-4184-a22a-244eb81366f9 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.197214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:08.198803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.201705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.201705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.201705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.201705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.202980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:47:08.202980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.202980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.228430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1127/2253] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:47:08.228430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.230853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.234773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1127/2254] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:47:08.236372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.252748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c06527-fcb7-478f-be30-9d50c1524850 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.253217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17976a-abe1-465e-b782-092e8aa6b9a6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.264058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.264964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.265338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.265703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] PATH_INFO: `/v3/projects/ed267396a0ff4ce4a88ebd72ea5577b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.267016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed267396a0ff4ce4a88ebd72ea5577b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.269063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.270472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.271547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.272086 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.272553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.273818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:47:08.274259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.277367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.297509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1128/2255] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:47:08.301483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.313438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d21eb-1eb3-4aca-b734-be219cdc6a6c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.321270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.321958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.322360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.322582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.323267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.325382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.351635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed267396a0ff4ce4a88ebd72ea5577b6', 'request_id': 'req-778651a7-dd0a-4a70-bd77-b61411e7394e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.362118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed267396a0ff4ce4a88ebd72ea5577b6', 'request_id': 'req-778651a7-dd0a-4a70-bd77-b61411e7394e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.363115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1129/2256] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:08.363970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.375470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed267396a0ff4ce4a88ebd72ea5577b6', 'request_id': 'req-778651a7-dd0a-4a70-bd77-b61411e7394e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.388231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] The token cache is being invalidate because project ed267396a0ff4ce4a88ebd72ea5577b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:08.388519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.389203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.390738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1128/2257] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:08 2022] DELETE /identity/v3/projects/ed267396a0ff4ce4a88ebd72ea5577b6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:08.391555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.393009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.421446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:08.434967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4cdb1d-a403-404a-8283-3d09813d9bbd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.441133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.441784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.442078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.442322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.442961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.445478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.454229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:08.473013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778651a7-dd0a-4a70-bd77-b61411e7394e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.478805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.479713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.480020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.480291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.480944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.483025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.523165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1129/2258] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:08.523165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.523165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.539856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71675665-98c7-466c-ba94-064fc8208680 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.544804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.545461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.545738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.546010 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] PATH_INFO: `/v3/users/53e7456d8e3449b889ff1de6d91ff45c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.547240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=53e7456d8e3449b889ff1de6d91ff45c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.549554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.602878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1130/2259] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/projects => generated 381 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:08.603866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.613883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53e7456d8e3449b889ff1de6d91ff45c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.621740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53e7456d8e3449b889ff1de6d91ff45c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.635810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53e7456d8e3449b889ff1de6d91ff45c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.636553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53e7456d8e3449b889ff1de6d91ff45c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.644084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53e7456d8e3449b889ff1de6d91ff45c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:08.650933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1130/2260] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:08 2022] DELETE /identity/v3/users/53e7456d8e3449b889ff1de6d91ff45c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:08.651998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.653241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.672393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1b410-af67-475f-a632-5a09968f570c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.735121 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6feb9-867f-4c46-ba9f-960d8baf2d3a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.741833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.742612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.742891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.743166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.744134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.746125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.760700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.761771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.762163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.762812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.764133 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.767399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.771908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1131/2261] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:08.776948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.784854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1131/2262] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:08.786353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.789072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954ea783-ac0e-4664-b797-8f8b60dedfda admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.795148 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.795403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.795595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.796252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.796918 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.798812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.801592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.816846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1132/2263] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:08.817782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.817893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19783990-b70b-43f3-9aed-68cee9908d7f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.823864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.824932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.824932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.825132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.825634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.827243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.831418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62a945-5182-4bc0-b7da-ba84e00b9007 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.838050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.838670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.838937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.839203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.839888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.841607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.854447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1132/2265] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:47:08.854611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1133/2265] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:08.858308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.859721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:08.870822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170e9d3-d6ea-4b3c-ae8c-399054ebce6c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.872431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561a77-9af4-4295-be8f-9c7322d84df9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:08.876276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e3cbc-c165-4048-9e39-e93ef0fa25fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.876962 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e3cbc-c165-4048-9e39-e93ef0fa25fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.877252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e3cbc-c165-4048-9e39-e93ef0fa25fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.877491 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e3cbc-c165-4048-9e39-e93ef0fa25fb admin admin] PATH_INFO: `/v3/projects/d0f3d305e6fb49caacbd005173c08172/users/81465400f2b5421392d78357db677173/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.878358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0479678-8e16-4f1e-962d-237a4fa450f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:08.879156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0479678-8e16-4f1e-962d-237a4fa450f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.879558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0479678-8e16-4f1e-962d-237a4fa450f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.879906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0479678-8e16-4f1e-962d-237a4fa450f9 admin admin] PATH_INFO: `/v3/projects/23f1d6189cfd4437a13a0908119c7ecb/users/36846c3821c14284893e25dd143d11e2/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.896499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e3cbc-c165-4048-9e39-e93ef0fa25fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f3d305e6fb49caacbd005173c08172, user_id=81465400f2b5421392d78357db677173, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.897817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0479678-8e16-4f1e-962d-237a4fa450f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23f1d6189cfd4437a13a0908119c7ecb, user_id=36846c3821c14284893e25dd143d11e2, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:08.899612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e3cbc-c165-4048-9e39-e93ef0fa25fb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.902747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0479678-8e16-4f1e-962d-237a4fa450f9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:08.907027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82,keystone:INSERT=8,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:08.918734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1133/2266] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:47:08 2022] PUT /identity/v3/projects/d0f3d305e6fb49caacbd005173c08172/users/81465400f2b5421392d78357db677173/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:47:08.920705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1134/2267] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:47:08 2022] PUT /identity/v3/projects/23f1d6189cfd4437a13a0908119c7ecb/users/36846c3821c14284893e25dd143d11e2/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:08.926022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.926272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.926599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.931352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:08.932157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:08.932840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:08.949540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:08.950801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:08.968710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] MFA Rules not processed for user `81465400f2b5421392d78357db677173`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:08.970489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.025290 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:09.052636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1134/2268] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:09.053601 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.055121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.063251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1135/2269] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:09.064239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.076715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.079185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bbed9-d5da-4743-8349-a24a473f6277 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.085211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.086014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.086289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.086562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] PATH_INFO: `/v3/projects/2d1e93c98fcf43f4b4e869dd9a4c058d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.087707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1e93c98fcf43f4b4e869dd9a4c058d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.089632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.150822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfbab3-84de-4ff1-8400-266d907d1588 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.175710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1e93c98fcf43f4b4e869dd9a4c058d', 'request_id': 'req-b2317578-2e99-42d6-82f1-8f1cef3ab247'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:09.181782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1e93c98fcf43f4b4e869dd9a4c058d', 'request_id': 'req-b2317578-2e99-42d6-82f1-8f1cef3ab247'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:09.188692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1e93c98fcf43f4b4e869dd9a4c058d', 'request_id': 'req-b2317578-2e99-42d6-82f1-8f1cef3ab247'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:09.209512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] The token cache is being invalidate because project 2d1e93c98fcf43f4b4e869dd9a4c058d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:09.210229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:09.211417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:09.214174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1135/2270] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:09 2022] DELETE /identity/v3/projects/2d1e93c98fcf43f4b4e869dd9a4c058d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:09.219933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.222101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.231967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.232666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.232943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.233210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.235221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.288841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2317578-2e99-42d6-82f1-8f1cef3ab247 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.293931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.294864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.295311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.295889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.298228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:09.298453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.318816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.321217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f3415-eed4-456e-ad03-60240d52a369 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.337137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1136/2271] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:09.338645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6730-6d9a-40ba-9b30-155c2b8de23b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.338991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6730-6d9a-40ba-9b30-155c2b8de23b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.339409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6730-6d9a-40ba-9b30-155c2b8de23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.363086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d49b6730-6d9a-40ba-9b30-155c2b8de23b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:09.364462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d49b6730-6d9a-40ba-9b30-155c2b8de23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.373483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1137/2272] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:09.398657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.400640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.415937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1136/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:09.448381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.448381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.448381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.478377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:09.480095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:09.492169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1138/2274] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:09.494766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.508179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5bd8e-58f5-496a-b1a4-f23ec1142cb9 service glance] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.513427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.513878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.514224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.514452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.515014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:47:09.515306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.517425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.530573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1137/2275] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:47:09.535154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.549031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052d779-f753-4667-ab26-cc0e72df05da None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.559474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.560708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.560708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.561242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.562527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.563841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.607388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1139/2276] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:47:09 2022] POST /identity/v3/projects => generated 369 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:47:09.612474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.627909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680d121-cbe9-4fb6-a9ef-93954b4f9e7e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.633733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:09.710364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.711220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.711667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.712064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.713139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.715536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.754238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1138/2277] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:47:09 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:09.759518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.788933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7010c2b-8858-4ac5-9fa4-03595e9445c1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.795558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.796885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.799082 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.799431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.801591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.805222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.824742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.825610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1140/2278] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:47:09.835794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f1d5-5173-43b1-934b-04340e371032 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.840307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b381687-3fe8-4cee-a7d8-d4f22e0e674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.840924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b381687-3fe8-4cee-a7d8-d4f22e0e674c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.841302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b381687-3fe8-4cee-a7d8-d4f22e0e674c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.841589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b381687-3fe8-4cee-a7d8-d4f22e0e674c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.842242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b381687-3fe8-4cee-a7d8-d4f22e0e674c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.843990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b381687-3fe8-4cee-a7d8-d4f22e0e674c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.864258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:09.865583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1139/2279] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:09.901174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d918bf9-28ce-4dde-93ff-51bf0bafdae3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:09.907599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:09.908309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.908609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.908899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] PATH_INFO: `/v3/projects/a4a93cabac6048138fdf52a1a8a4b89d/users/4444b5f2f764422a94cc1cda20d2828b/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:09.932815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a93cabac6048138fdf52a1a8a4b89d, user_id=4444b5f2f764422a94cc1cda20d2828b, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:09.938415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:09.972094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1141/2280] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:47:09 2022] PUT /identity/v3/projects/a4a93cabac6048138fdf52a1a8a4b89d/users/4444b5f2f764422a94cc1cda20d2828b/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:47:09.977332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52ca7b-55f9-4165-8ea4-85077b921337 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:09.977668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52ca7b-55f9-4165-8ea4-85077b921337 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:09.977860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52ca7b-55f9-4165-8ea4-85077b921337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:10.025487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7f52ca7b-55f9-4165-8ea4-85077b921337 None None] MFA Rules not processed for user `4444b5f2f764422a94cc1cda20d2828b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:10.030686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7f52ca7b-55f9-4165-8ea4-85077b921337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:10.196803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1140/2281] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:10.205135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:10.282369 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181e41-4e2e-4ddc-99ea-fa8354827be8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:10.287650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:10.288580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:10.289002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:10.289583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:10.300557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:10.301832 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:10.314233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1142/2282] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:10.366655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b67f81-c177-4c14-b257-f8a023b21918 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:10.366973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b67f81-c177-4c14-b257-f8a023b21918 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:10.368113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b67f81-c177-4c14-b257-f8a023b21918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:10.416947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d6b67f81-c177-4c14-b257-f8a023b21918 None None] MFA Rules not processed for user `36846c3821c14284893e25dd143d11e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:10.419458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6b67f81-c177-4c14-b257-f8a023b21918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:10.588827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1141/2283] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:10.720825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:10.722278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:10.789359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba448d-c4e2-49c8-93ba-10595818066b service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:10.792998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:10.793612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:10.793888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:10.794244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:10.803997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:10.805550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:10.815270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1143/2284] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:12.267801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5f9b-d0f5-4aa6-b16c-50db1818ea39 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:12.267801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5f9b-d0f5-4aa6-b16c-50db1818ea39 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:12.267801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d5f9b-d0f5-4aa6-b16c-50db1818ea39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:12.277393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:12.284941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b06d5f9b-d0f5-4aa6-b16c-50db1818ea39 None None] MFA Rules not processed for user `36846c3821c14284893e25dd143d11e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:12.286553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b06d5f9b-d0f5-4aa6-b16c-50db1818ea39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:12.295070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1142/2285] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:15.722358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:15.723848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:15.757768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:15.796767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd8f26-615b-4d91-9e75-82b96baacf16 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:15.801281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:15.801904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:15.802180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:15.802478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] PATH_INFO: `/v3/users/6130f3cd53f94107a6f09330e538e7d3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:15.803600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6130f3cd53f94107a6f09330e538e7d3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:15.805560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:15.855024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6130f3cd53f94107a6f09330e538e7d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:15.866162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6130f3cd53f94107a6f09330e538e7d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:15.885785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6130f3cd53f94107a6f09330e538e7d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:15.886558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6130f3cd53f94107a6f09330e538e7d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:15.893547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6130f3cd53f94107a6f09330e538e7d3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:15.899762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1144/2286] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:15 2022] DELETE /identity/v3/users/6130f3cd53f94107a6f09330e538e7d3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:15.904196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:15.904651 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:15.904741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:15.925469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:15.926667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:16.001037 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1143/2287] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:16.017837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:16.019428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:16.105316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072c4a-5810-4521-b479-1aabca619c9e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:16.112696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87a664-9351-4b17-a304-78182649d55d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:16.113732 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87a664-9351-4b17-a304-78182649d55d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:16.114009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87a664-9351-4b17-a304-78182649d55d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:16.114279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87a664-9351-4b17-a304-78182649d55d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:16.124516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87a664-9351-4b17-a304-78182649d55d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:16.126230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87a664-9351-4b17-a304-78182649d55d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:16.138238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1145/2288] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:16.854988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:16.857926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:16.872588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5153cb-ff6e-4750-85be-07dde7f940d4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:16.878756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:16.879546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:16.879972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:16.880205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] PATH_INFO: `/v3/projects/771c30a1806443868a248befdcaa3d21` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:16.883069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771c30a1806443868a248befdcaa3d21)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:16.887175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:16.963185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771c30a1806443868a248befdcaa3d21', 'request_id': 'req-cee7bde7-e9b9-4549-881a-f3e7354fff72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:16.975242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771c30a1806443868a248befdcaa3d21', 'request_id': 'req-cee7bde7-e9b9-4549-881a-f3e7354fff72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:16.982970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771c30a1806443868a248befdcaa3d21', 'request_id': 'req-cee7bde7-e9b9-4549-881a-f3e7354fff72'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:16.999244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] The token cache is being invalidate because project 771c30a1806443868a248befdcaa3d21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:16.999526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:17.000309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-cee7bde7-e9b9-4549-881a-f3e7354fff72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:17.001836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1144/2289] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:16 2022] DELETE /identity/v3/projects/771c30a1806443868a248befdcaa3d21 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:17.007164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.007679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.007785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.029371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:17.030691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:17.118665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:17.143853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1146/2290] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:17.149843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.150186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.150558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.178667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:17.180162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:17.188961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1145/2291] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:47:17.193745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.205783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d92454-ed84-459c-b8d0-40d1eba932a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.212730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.213364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.213642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.214003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.214622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:47:17.214936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.216731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.240468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1147/2292] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:47:17.242115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.252695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292f56e-43f7-41df-9f1d-8bbd90095513 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.258363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.258987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.259323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.259719 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.260394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.262316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.307886 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1146/2293] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:47:17 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:17.313403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.324469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67f65d-e01a-4f02-a16f-1268e1a60740 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.413553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.414191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.414470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.414755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.415372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.417500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.464420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1148/2294] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:47:17 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:17.465253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.476805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5ed79-5956-4b9d-bc6c-97b473fab1db admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.484772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.485775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.486094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.486360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.487306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.489067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.504712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1147/2295] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:47:17.507711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.530063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef70c19-6cf2-4658-81de-b073d9ef46d8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.534778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a61aa9-c5c9-4995-b47f-56d30fbf0be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.535346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61aa9-c5c9-4995-b47f-56d30fbf0be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.535625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61aa9-c5c9-4995-b47f-56d30fbf0be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.536263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a61aa9-c5c9-4995-b47f-56d30fbf0be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.537716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61aa9-c5c9-4995-b47f-56d30fbf0be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.540189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a61aa9-c5c9-4995-b47f-56d30fbf0be3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.554363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1149/2296] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:17.560404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.579220 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2762b9-5eac-4552-83ad-c33191e4c444 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.589111 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.589797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.590191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.590724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] PATH_INFO: `/v3/projects/7d62e805c3154d9497d2b0c2135b2c5d/users/72172eec4ed6442dbc9e93618f938f97/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.609806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d62e805c3154d9497d2b0c2135b2c5d, user_id=72172eec4ed6442dbc9e93618f938f97, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.612776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.632030 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1148/2297] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:47:17 2022] PUT /identity/v3/projects/7d62e805c3154d9497d2b0c2135b2c5d/users/72172eec4ed6442dbc9e93618f938f97/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:17.643018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826708-adfa-4273-a038-4d5991887231 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.643611 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826708-adfa-4273-a038-4d5991887231 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.644103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826708-adfa-4273-a038-4d5991887231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.686980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-88826708-adfa-4273-a038-4d5991887231 None None] MFA Rules not processed for user `72172eec4ed6442dbc9e93618f938f97`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:17.688356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-88826708-adfa-4273-a038-4d5991887231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:17.782585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1150/2298] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:17.794876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:17.797361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:17.830101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:17.893479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d332e-cf14-4aa5-80f3-df30cc486984 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:17.897479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207293f-02f1-4641-9cb6-31c6037b1a6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:17.898094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207293f-02f1-4641-9cb6-31c6037b1a6b service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:17.898370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207293f-02f1-4641-9cb6-31c6037b1a6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:17.898635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207293f-02f1-4641-9cb6-31c6037b1a6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:17.915641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207293f-02f1-4641-9cb6-31c6037b1a6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:17.917566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207293f-02f1-4641-9cb6-31c6037b1a6b service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:17.928082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1149/2299] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:27.775793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:27.923016 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=17 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:28.610076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826708-adfa-4273-a038-4d5991887231 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:28.672125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826708-adfa-4273-a038-4d5991887231 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:28.676591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:28.677217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:28.677485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:28.677753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] PATH_INFO: `/v3/users/72172eec4ed6442dbc9e93618f938f97` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:28.679404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] RBAC: Authorizing `identity:delete_user(user_id=72172eec4ed6442dbc9e93618f938f97)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:28.681390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:28.729048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72172eec4ed6442dbc9e93618f938f97'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:28.736850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72172eec4ed6442dbc9e93618f938f97'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:28.750623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72172eec4ed6442dbc9e93618f938f97'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:28.751433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72172eec4ed6442dbc9e93618f938f97'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:28.759682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72172eec4ed6442dbc9e93618f938f97'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:28.766183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1151/2300] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:28 2022] DELETE /identity/v3/users/72172eec4ed6442dbc9e93618f938f97 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:28.770648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:28.770925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:28.771192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:28.790565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:28.792042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:28.855216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1150/2301] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:28.864866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:28.866947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:28.931813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8c599-893d-4b5e-9a81-27dc75e7341f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:28.935515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830602f-3b63-433a-9b73-4827bdb5927e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:28.936156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830602f-3b63-433a-9b73-4827bdb5927e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:28.936458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830602f-3b63-433a-9b73-4827bdb5927e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:28.936730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830602f-3b63-433a-9b73-4827bdb5927e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:28.947649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830602f-3b63-433a-9b73-4827bdb5927e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:28.949254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830602f-3b63-433a-9b73-4827bdb5927e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:28.961439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1152/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:29.251803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.253287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:29.265993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29a0d8-6954-4f83-b7fc-190aada389f2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.270931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.271585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.272517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.272790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] PATH_INFO: `/v3/projects/7d62e805c3154d9497d2b0c2135b2c5d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.273915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d62e805c3154d9497d2b0c2135b2c5d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.275870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.326167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d62e805c3154d9497d2b0c2135b2c5d', 'request_id': 'req-ef0f1864-278c-4f1a-a50e-78319138cd45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:29.332058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d62e805c3154d9497d2b0c2135b2c5d', 'request_id': 'req-ef0f1864-278c-4f1a-a50e-78319138cd45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:29.339036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d62e805c3154d9497d2b0c2135b2c5d', 'request_id': 'req-ef0f1864-278c-4f1a-a50e-78319138cd45'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:29.355421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] The token cache is being invalidate because project 7d62e805c3154d9497d2b0c2135b2c5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:29.355710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:29.356454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ef0f1864-278c-4f1a-a50e-78319138cd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:29.357871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1151/2303] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:29 2022] DELETE /identity/v3/projects/7d62e805c3154d9497d2b0c2135b2c5d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:29.362961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.363199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.363471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.383022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:29.384277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:29.403690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:29.452127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1153/2304] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:29.462280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.462558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.462885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.483326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:29.484567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:29.493885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1152/2305] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:29.498150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.509389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453290d0-3153-4ca5-8d66-55510b455050 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.514275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.514969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.515250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.515522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.516234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:47:29.516540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.518159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.534869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1154/2306] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:47:29.540191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.552827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86a4d3-1986-4027-ac65-05bce3bf21dc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.555727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.556392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.556740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.557168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.557638 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.559542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.599120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1153/2307] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:47:29 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:29.601367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.612283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19702ace-2b36-411d-be3b-f09581fdaaa8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.674672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.675279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.675559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.675859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.676844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.678875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.705269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1155/2308] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:47:29 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:29.707375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.719310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a79b7-4bf8-46fe-84ef-9b0db6cce286 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.727470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.728258 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.728584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.728922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.729927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.731593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.745255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1154/2309] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:29.746827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.759337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c28bc0-773e-4698-b564-088ec335d00b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.763863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddff5e0-d048-4891-9009-8e70c5ee3499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.764550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff5e0-d048-4891-9009-8e70c5ee3499 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.764835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff5e0-d048-4891-9009-8e70c5ee3499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.765109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddff5e0-d048-4891-9009-8e70c5ee3499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.765760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddff5e0-d048-4891-9009-8e70c5ee3499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.767349 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddff5e0-d048-4891-9009-8e70c5ee3499 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.780080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1156/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:29.784179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.794264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942680c-a1c4-4206-88e8-090ec8630ed4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:29.798887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:29.799537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.799978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.800124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] PATH_INFO: `/v3/projects/fd1e169648bf40c2a95976598a973864/users/bc972363eaaa43e685923fe12e3ba2c3/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.814751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd1e169648bf40c2a95976598a973864, user_id=bc972363eaaa43e685923fe12e3ba2c3, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:29.817401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:29.831831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1155/2311] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:47:29 2022] PUT /identity/v3/projects/fd1e169648bf40c2a95976598a973864/users/bc972363eaaa43e685923fe12e3ba2c3/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:29.842701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:29.842986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:29.843331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:29.870920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] MFA Rules not processed for user `bc972363eaaa43e685923fe12e3ba2c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:29.872090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:29.946365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1157/2312] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:29 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:29.957401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:29.959191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:29.981632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:30.025217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02baf2e-e191-402f-878c-3146a9ef8b3a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:30.029014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f259f8c-62f2-4ebb-907e-e8b47a30cb8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:30.029631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f259f8c-62f2-4ebb-907e-e8b47a30cb8f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:30.029896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f259f8c-62f2-4ebb-907e-e8b47a30cb8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:30.030166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f259f8c-62f2-4ebb-907e-e8b47a30cb8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:30.042776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f259f8c-62f2-4ebb-907e-e8b47a30cb8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:30.044512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f259f8c-62f2-4ebb-907e-e8b47a30cb8f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:30.057009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1156/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:39.942565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:40.050916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=17 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:47.438043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:47.509456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36930a-8b11-4d85-a436-c7814c5e59cb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:47.515283 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:47.515960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:47.516231 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:47.516497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] PATH_INFO: `/v3/users/bc972363eaaa43e685923fe12e3ba2c3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:47.517612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc972363eaaa43e685923fe12e3ba2c3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:47.519561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:47.575504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc972363eaaa43e685923fe12e3ba2c3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:47.584168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc972363eaaa43e685923fe12e3ba2c3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:47.599194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc972363eaaa43e685923fe12e3ba2c3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:47.599962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc972363eaaa43e685923fe12e3ba2c3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:47.607724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc972363eaaa43e685923fe12e3ba2c3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:47.614765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1158/2314] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:47 2022] DELETE /identity/v3/users/bc972363eaaa43e685923fe12e3ba2c3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:47.620912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:47.621444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:47.621444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:47.643484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:47.644742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:47.724230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1157/2315] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:47.734276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:47.735862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:47.808448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b393d3-2156-4e8a-9290-b57821481484 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:47.811384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:47.813152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:47.813152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:47.813152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:47.826197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:47.828229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:47.840460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1159/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:48.209759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:48.211222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:48.232698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3281d-6bb6-4084-9d04-cf8976cb4ac4 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:48.245264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:48.245264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:48.245264 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:48.245744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] PATH_INFO: `/v3/projects/fd1e169648bf40c2a95976598a973864` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:48.247867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd1e169648bf40c2a95976598a973864)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:48.251700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:48.331384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd1e169648bf40c2a95976598a973864', 'request_id': 'req-448996b9-08ce-4eab-8b7b-6085e111f146'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:48.337419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd1e169648bf40c2a95976598a973864', 'request_id': 'req-448996b9-08ce-4eab-8b7b-6085e111f146'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:48.344444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd1e169648bf40c2a95976598a973864', 'request_id': 'req-448996b9-08ce-4eab-8b7b-6085e111f146'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:48.361990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] The token cache is being invalidate because project fd1e169648bf40c2a95976598a973864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:48.362298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:48.363345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-448996b9-08ce-4eab-8b7b-6085e111f146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:48.364496 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1158/2317] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:48 2022] DELETE /identity/v3/projects/fd1e169648bf40c2a95976598a973864 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:52.958185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:52.964969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:52.992476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323eb228-d379-48bd-a59a-035cf229ea3b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:53.016258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:53.058153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:53.058698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.058974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.059291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] PATH_INFO: `/v3/users/81465400f2b5421392d78357db677173` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.060380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81465400f2b5421392d78357db677173)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:53.062321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:53.115140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81465400f2b5421392d78357db677173'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.123035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81465400f2b5421392d78357db677173'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.135373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81465400f2b5421392d78357db677173'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.136123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81465400f2b5421392d78357db677173'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.143204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81465400f2b5421392d78357db677173'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.151976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1160/2318] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:47:52 2022] DELETE /identity/v3/users/81465400f2b5421392d78357db677173 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:53.156615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.157089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.157192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.177179 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:53.178462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:53.238797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1159/2319] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:53.249166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:53.250557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:53.310792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8f3ad-3974-4047-8c0c-2a14d1ffcf79 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:53.314365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecd301-db95-4ba1-a44e-4505526b3016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:53.314972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd301-db95-4ba1-a44e-4505526b3016 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.315243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd301-db95-4ba1-a44e-4505526b3016 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.315509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecd301-db95-4ba1-a44e-4505526b3016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.326398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecd301-db95-4ba1-a44e-4505526b3016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:53.327872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecd301-db95-4ba1-a44e-4505526b3016 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:53.337772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1161/2320] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:53.617605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:53.619189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:53.634758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a9162-1aea-4711-b2c3-699e7b86747f None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:53.640239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:53.640883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.641156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.641490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] PATH_INFO: `/v3/projects/d0f3d305e6fb49caacbd005173c08172` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.642611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f3d305e6fb49caacbd005173c08172)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:53.644517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:53.668183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:53.698766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f3d305e6fb49caacbd005173c08172', 'request_id': 'req-0e2c3b67-8e63-444a-b48a-321c6ec776bb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.704293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f3d305e6fb49caacbd005173c08172', 'request_id': 'req-0e2c3b67-8e63-444a-b48a-321c6ec776bb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.711052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f3d305e6fb49caacbd005173c08172', 'request_id': 'req-0e2c3b67-8e63-444a-b48a-321c6ec776bb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.724998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] The token cache is being invalidate because project d0f3d305e6fb49caacbd005173c08172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:47:53.725325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.726004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e2c3b67-8e63-444a-b48a-321c6ec776bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:47:53.727495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1160/2321] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:47:53 2022] DELETE /identity/v3/projects/d0f3d305e6fb49caacbd005173c08172 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:53.732653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.732972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.733247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.752631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:53.753833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:53.796587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:53.822483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1162/2322] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:53.830219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.830501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.830781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.857490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:53.858720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:53.867893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1161/2323] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:53.872275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:53.882606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78d715-26e5-4898-8745-69ffd9d2c15f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:53.887396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:53.888050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.888332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.888606 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.889251 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:47:53.889556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:53.891156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:53.903995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1163/2324] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:47:53.908275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:53.918393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779b5b3-28b4-4f95-9318-75adae60bd99 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:53.923113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:53.923725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:53.924256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:53.924557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:53.925139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:53.929487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:53.962338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1162/2325] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:47:53 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:53.968575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:53.980115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31541936-a0dd-49f8-b55a-f28be7abded4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:54.046785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:54.047067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:54.047312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:54.047497 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:54.048142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:54.050312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:54.079322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1164/2326] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:47:53 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:47:54.083780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:54.095328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2691df0-88a8-4f5a-a626-7c4ad309aca9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:54.100907 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:54.101544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:54.101841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:54.102112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:54.102754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:54.104691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:54.117285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1163/2327] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:47:54.121949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:54.135595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e1033-af8e-48d1-bb3b-18d1500ff286 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:54.140461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9258e9-c9e9-401a-b36c-753b4e3abc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:54.141201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9258e9-c9e9-401a-b36c-753b4e3abc7a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:54.141490 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9258e9-c9e9-401a-b36c-753b4e3abc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:54.141759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9258e9-c9e9-401a-b36c-753b4e3abc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:54.142403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9258e9-c9e9-401a-b36c-753b4e3abc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:54.144087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9258e9-c9e9-401a-b36c-753b4e3abc7a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:54.158510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1165/2328] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:47:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:47:54.163296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:54.173979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be785d-83dd-4ec1-a9b9-1691b1d6d859 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:54.178769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:54.179418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:54.179691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:54.179986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] PATH_INFO: `/v3/projects/2aa6bd99403b4421acb8bd714c9450c6/users/d10c1b55b5c04ba380ed8ffdaa462bc7/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:54.198313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa6bd99403b4421acb8bd714c9450c6, user_id=d10c1b55b5c04ba380ed8ffdaa462bc7, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:54.201001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:54.216992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1164/2329] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:47:54 2022] PUT /identity/v3/projects/2aa6bd99403b4421acb8bd714c9450c6/users/d10c1b55b5c04ba380ed8ffdaa462bc7/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:47:54.222162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe998-06b8-41a9-a47e-91e34f8524f0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:54.222447 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe998-06b8-41a9-a47e-91e34f8524f0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:54.222723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfe998-06b8-41a9-a47e-91e34f8524f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:54.242859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-92bfe998-06b8-41a9-a47e-91e34f8524f0 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:54.244100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-92bfe998-06b8-41a9-a47e-91e34f8524f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:54.306625 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1166/2330] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:54.321515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:54.323301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:54.385345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1fe951-1ccb-4e01-b283-f7c5b48db650 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:54.389501 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:54.390115 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:54.390387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:54.390661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:54.401352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:54.402772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:54.412340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1165/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:55.103203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7639a4c-07b9-494d-b478-73b9e87ff575 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:55.103697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7639a4c-07b9-494d-b478-73b9e87ff575 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:55.103827 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7639a4c-07b9-494d-b478-73b9e87ff575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:55.147307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d7639a4c-07b9-494d-b478-73b9e87ff575 None None] MFA Rules not processed for user `d10c1b55b5c04ba380ed8ffdaa462bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:55.149549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d7639a4c-07b9-494d-b478-73b9e87ff575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:55.207559 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:55.254225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1167/2332] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:47:55.267279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:47:55.268990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:55.349670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:47:55.356912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47999a73-4a3a-4f7a-a252-145449afd68e service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:47:55.364198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:47:55.365524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:55.366652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:55.366931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:55.379362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:47:55.380637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:47:55.391602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1166/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:47:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:47:57.874097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e96b7d-7a5d-4249-8829-044744e4abac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:47:57.874623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e96b7d-7a5d-4249-8829-044744e4abac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:47:57.874751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e96b7d-7a5d-4249-8829-044744e4abac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:47:57.896182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-21e96b7d-7a5d-4249-8829-044744e4abac None None] MFA Rules not processed for user `d10c1b55b5c04ba380ed8ffdaa462bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:47:57.897933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-21e96b7d-7a5d-4249-8829-044744e4abac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:47:57.904834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1168/2334] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:47:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:01.912415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:01.914387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:01.927710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d79549-1d49-42bc-8f62-e1d10892adc5 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:01.932905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:01.933543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:01.933815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:01.934070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] PATH_INFO: `/v3/users/aa8ebc429a4a45ae8742252218799f3b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:01.935239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8ebc429a4a45ae8742252218799f3b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:01.937120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:01.998299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8ebc429a4a45ae8742252218799f3b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.015249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8ebc429a4a45ae8742252218799f3b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.025289 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8ebc429a4a45ae8742252218799f3b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.028518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8ebc429a4a45ae8742252218799f3b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.043108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8ebc429a4a45ae8742252218799f3b'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.051839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1167/2335] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:48:01 2022] DELETE /identity/v3/users/aa8ebc429a4a45ae8742252218799f3b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:02.057333 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.057845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.057990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.082115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:02.083396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:02.164798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1169/2336] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:02.175410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:02.176948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:02.255516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29eaf4-0f70-406b-9b26-7390576b48f9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:02.259894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d79c26-3844-4164-9ad2-a52b726338a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:02.260508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d79c26-3844-4164-9ad2-a52b726338a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.260791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d79c26-3844-4164-9ad2-a52b726338a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.261056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d79c26-3844-4164-9ad2-a52b726338a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.270928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d79c26-3844-4164-9ad2-a52b726338a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:02.272630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d79c26-3844-4164-9ad2-a52b726338a5 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:02.282830 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1168/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:02.597498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:02.600531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:02.618123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73476ec6-a57e-459f-9581-2c04086ee605 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:02.623396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:02.624080 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.624355 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.624637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] PATH_INFO: `/v3/projects/c282713d51e74b6b87a7c0024cc18eb8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.625802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c282713d51e74b6b87a7c0024cc18eb8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:02.627920 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:02.680405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c282713d51e74b6b87a7c0024cc18eb8', 'request_id': 'req-165cb38f-62f6-44eb-aea6-3925345b3266'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.688935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c282713d51e74b6b87a7c0024cc18eb8', 'request_id': 'req-165cb38f-62f6-44eb-aea6-3925345b3266'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.694953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c282713d51e74b6b87a7c0024cc18eb8', 'request_id': 'req-165cb38f-62f6-44eb-aea6-3925345b3266'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.707984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] The token cache is being invalidate because project c282713d51e74b6b87a7c0024cc18eb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:48:02.708338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.708851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-165cb38f-62f6-44eb-aea6-3925345b3266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:02.710474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1170/2338] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:48:02 2022] DELETE /identity/v3/projects/c282713d51e74b6b87a7c0024cc18eb8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:02.715611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.715936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.716217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.737711 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:02.738930 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:02.804839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1169/2339] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:48:02.814789 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.815071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.815344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.839416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:02.840751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:02.849663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1171/2340] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:02.854317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:02.861036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:02.865374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029dc54-9200-494a-ac64-9903edf5f887 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:02.872936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:02.873991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.874370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.874644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.875350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:48:02.876039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:02.878202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:02.891166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1170/2341] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:48:02.895972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:02.906838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3e381-2b88-4401-9f8d-d359e6ac8629 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:02.911674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:02.912317 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:02.913204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:02.913485 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:02.914136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:02.916072 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:02.945551 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1172/2342] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:48:02 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:48:02.946919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:02.958141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7aa2b2-6a62-49db-942e-28fb7b365220 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:03.029549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:03.030174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:03.030442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:03.030707 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:03.031373 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:03.033426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:03.057572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1171/2343] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:48:02 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:48:03.061907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:03.072826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba1046-9866-4482-b5db-0fd46bed3de9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:03.077725 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:03.078331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:03.078623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:03.078871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:03.079521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:03.081431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:03.092401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1173/2344] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:48:03.097157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:03.108715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34116d10-2ceb-4762-b2fa-bb36e4c6d9bb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:03.113485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c4f248-1f8c-43a5-bb07-dba7ec349ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:03.114089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4f248-1f8c-43a5-bb07-dba7ec349ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:03.114367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4f248-1f8c-43a5-bb07-dba7ec349ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:03.114635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c4f248-1f8c-43a5-bb07-dba7ec349ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:03.115282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4f248-1f8c-43a5-bb07-dba7ec349ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:03.117306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c4f248-1f8c-43a5-bb07-dba7ec349ddd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:03.128583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1172/2345] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:48:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:48:03.135830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:03.146549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c38f84-b927-4256-ab0f-2a0721af4aca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:03.151844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:03.152495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:03.152771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:03.153042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] PATH_INFO: `/v3/projects/d89c15d74a584be3b5e44f74ed50eea3/users/c05718633def4ad6822728767580b810/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:03.168107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89c15d74a584be3b5e44f74ed50eea3, user_id=c05718633def4ad6822728767580b810, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:03.170775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:03.172893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:03.185214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1174/2346] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:48:03 2022] PUT /identity/v3/projects/d89c15d74a584be3b5e44f74ed50eea3/users/c05718633def4ad6822728767580b810/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:03.195702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d065-0ac7-440a-9f26-7780a8008798 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:03.196011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d065-0ac7-440a-9f26-7780a8008798 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:03.196284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518d065-0ac7-440a-9f26-7780a8008798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:03.224442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a518d065-0ac7-440a-9f26-7780a8008798 None None] MFA Rules not processed for user `c05718633def4ad6822728767580b810`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:03.226141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a518d065-0ac7-440a-9f26-7780a8008798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:03.291857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1173/2347] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:03.303210 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:03.305745 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:03.368165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54fcb0-d654-4f02-aee9-e3e870d589c0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:03.372033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:03.372679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:03.372954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:03.373224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:03.383478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:03.385183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:03.401036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1175/2348] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:05.901316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1f14-c942-4c6a-a590-4f55e8f9f42f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:05.901852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1f14-c942-4c6a-a590-4f55e8f9f42f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:05.901852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe1f14-c942-4c6a-a590-4f55e8f9f42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:05.903206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:05.905036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:05.922091 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-fdfe1f14-c942-4c6a-a590-4f55e8f9f42f None None] MFA Rules not processed for user `c05718633def4ad6822728767580b810`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:05.923310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fdfe1f14-c942-4c6a-a590-4f55e8f9f42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:05.931556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1174/2349] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:05.977120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62ebaf-4597-4104-874f-5f3f947d3936 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:05.982066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:05.982719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:05.982971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:05.983239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] PATH_INFO: `/v3/users/4444b5f2f764422a94cc1cda20d2828b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:05.984412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4444b5f2f764422a94cc1cda20d2828b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:05.986489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:06.047044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4444b5f2f764422a94cc1cda20d2828b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.056767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4444b5f2f764422a94cc1cda20d2828b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.070125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4444b5f2f764422a94cc1cda20d2828b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.070862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4444b5f2f764422a94cc1cda20d2828b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.077936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4444b5f2f764422a94cc1cda20d2828b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.083764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1176/2350] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:48:05 2022] DELETE /identity/v3/users/4444b5f2f764422a94cc1cda20d2828b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:06.088783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:06.089059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:06.089322 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:06.108807 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:06.109896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:06.161294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:06.175837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1175/2351] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:48:06.185775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:06.187222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:06.208925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:06.264707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b97f76-6c80-4b2f-9ac6-ae6fb7569dae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:06.269828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1df5d-b23c-4ff8-8b4b-1599eb472cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:06.270588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1df5d-b23c-4ff8-8b4b-1599eb472cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:06.270929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1df5d-b23c-4ff8-8b4b-1599eb472cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:06.271261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1df5d-b23c-4ff8-8b4b-1599eb472cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:06.282025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1df5d-b23c-4ff8-8b4b-1599eb472cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:06.283619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1df5d-b23c-4ff8-8b4b-1599eb472cd9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:06.294578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1177/2352] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:06.628177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:06.630875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:06.644082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc90ac2-b8cc-40c1-94cb-a423def0665a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:06.651298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:06.652454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:06.653053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:06.653615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] PATH_INFO: `/v3/projects/a4a93cabac6048138fdf52a1a8a4b89d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:06.654854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a93cabac6048138fdf52a1a8a4b89d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:06.656953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:06.713035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a93cabac6048138fdf52a1a8a4b89d', 'request_id': 'req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.719270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a93cabac6048138fdf52a1a8a4b89d', 'request_id': 'req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.726397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a93cabac6048138fdf52a1a8a4b89d', 'request_id': 'req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.740454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] The token cache is being invalidate because project a4a93cabac6048138fdf52a1a8a4b89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:48:06.741420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.742078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:06.743669 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1176/2353] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:48:06 2022] DELETE /identity/v3/projects/a4a93cabac6048138fdf52a1a8a4b89d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:16.053387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3acaaa-9b42-4855-b26e-3074f282a257 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:16.055816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3acaaa-9b42-4855-b26e-3074f282a257 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:16.055816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3acaaa-9b42-4855-b26e-3074f282a257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:16.076519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a3acaaa-9b42-4855-b26e-3074f282a257 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:16.077835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a3acaaa-9b42-4855-b26e-3074f282a257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:16.181926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1178/2354] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:16.231669 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:16.231669 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:16.316826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b69bad-5287-4bdd-b9c3-b05961b7e7a8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:16.323891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:16.324636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:16.324815 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:16.324961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:16.334022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:16.336307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:16.351775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1177/2355] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:22.109262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84307-5909-4d9c-a0c9-3167436c9512 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:22.109791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84307-5909-4d9c-a0c9-3167436c9512 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:22.109791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f84307-5909-4d9c-a0c9-3167436c9512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:22.129180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-12f84307-5909-4d9c-a0c9-3167436c9512 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:22.130388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12f84307-5909-4d9c-a0c9-3167436c9512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:22.138723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1179/2356] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:22.159252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:22.160655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:22.175233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763c1ef-8f16-42c1-bb7c-4782cb3aedc4 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:22.180859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b57c75-697f-415f-b9fd-2153d23dbecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:22.181384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57c75-697f-415f-b9fd-2153d23dbecd service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:22.181671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57c75-697f-415f-b9fd-2153d23dbecd service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:22.181937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b57c75-697f-415f-b9fd-2153d23dbecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:22.190277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b57c75-697f-415f-b9fd-2153d23dbecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:22.191895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b57c75-697f-415f-b9fd-2153d23dbecd service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:22.202516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1178/2357] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:27.218711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336433d-65d7-44df-8a7f-bbe249cff50a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:27.219170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336433d-65d7-44df-8a7f-bbe249cff50a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:27.219512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336433d-65d7-44df-8a7f-bbe249cff50a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:27.220997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1180/2358] 10.222.0.52 () {54 vars in 961 bytes} [Fri Oct 28 09:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:48:27.226940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340566d1-3c3f-40a0-8575-256f4c415487 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:27.227398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340566d1-3c3f-40a0-8575-256f4c415487 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:27.227530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340566d1-3c3f-40a0-8575-256f4c415487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:27.247551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-340566d1-3c3f-40a0-8575-256f4c415487 None None] MFA Rules not processed for user `daf577de783b4e8a9bc9c81dd2a2bd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:27.248763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-340566d1-3c3f-40a0-8575-256f4c415487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:27.300818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:27.322774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1179/2359] 10.222.0.52 () {58 vars in 1058 bytes} [Fri Oct 28 09:48:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:27.330374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336433d-65d7-44df-8a7f-bbe249cff50a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:27.344316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336433d-65d7-44df-8a7f-bbe249cff50a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:27.352622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:27.353233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:27.353509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:27.353776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:27.355491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:27.368460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:27.370083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:27.381670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1181/2360] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:31.579577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877fc23-50df-4753-a263-88da8ee729ea None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:31.580130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877fc23-50df-4753-a263-88da8ee729ea None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:31.580130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877fc23-50df-4753-a263-88da8ee729ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:31.611014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b877fc23-50df-4753-a263-88da8ee729ea None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:31.612296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b877fc23-50df-4753-a263-88da8ee729ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:31.621581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1180/2361] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:31.660821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:31.684022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079f1ee-322f-427c-9db1-bd1d87701821 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:31.690126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a73163-c4d5-43d4-86e6-ba88d1e59885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:31.690748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a73163-c4d5-43d4-86e6-ba88d1e59885 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:31.691022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a73163-c4d5-43d4-86e6-ba88d1e59885 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:31.691293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a73163-c4d5-43d4-86e6-ba88d1e59885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:31.700655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a73163-c4d5-43d4-86e6-ba88d1e59885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:31.702162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a73163-c4d5-43d4-86e6-ba88d1e59885 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:31.712517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1182/2362] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:32.282536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:32.283024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:32.283328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:32.339473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] MFA Rules not processed for user `cfe6eb38e8df492daaef5755ddea7773`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:32.341535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:32.413044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1181/2363] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:37.753326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c345e-e2ec-4bbd-a8e7-6f809fadc47e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:37.753853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c345e-e2ec-4bbd-a8e7-6f809fadc47e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:37.754004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c345e-e2ec-4bbd-a8e7-6f809fadc47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:37.761907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=62 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:37.776776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-584c345e-e2ec-4bbd-a8e7-6f809fadc47e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:37.777999 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-584c345e-e2ec-4bbd-a8e7-6f809fadc47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:37.787189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1183/2364] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:48:37.828866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:37.843121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a997-5f73-4074-8987-24664115a862 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:37.848683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dceae-54d9-415e-bdc2-9ddc81f7795b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:37.849295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dceae-54d9-415e-bdc2-9ddc81f7795b service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:37.849570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dceae-54d9-415e-bdc2-9ddc81f7795b service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:37.849840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dceae-54d9-415e-bdc2-9ddc81f7795b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:37.859414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dceae-54d9-415e-bdc2-9ddc81f7795b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:37.861151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dceae-54d9-415e-bdc2-9ddc81f7795b service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:37.873751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1182/2365] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:38.325923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:38.326209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:38.326482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:38.346029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] MFA Rules not processed for user `d10c1b55b5c04ba380ed8ffdaa462bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:38.347234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:38.406203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1184/2366] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:42.242044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0d93-d566-4d3b-9dd8-90f8fbf79088 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:42.242563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0d93-d566-4d3b-9dd8-90f8fbf79088 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:42.242691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa0d93-d566-4d3b-9dd8-90f8fbf79088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:42.302758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a3aa0d93-d566-4d3b-9dd8-90f8fbf79088 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:42.304237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a3aa0d93-d566-4d3b-9dd8-90f8fbf79088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:42.368364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1183/2367] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:48:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:48:42.420070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:42.421546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:42.496337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d920707-f420-41a6-8d07-db2203691b53 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:42.501290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84496595-6479-42bb-9239-2dd43325fe3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:42.501901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84496595-6479-42bb-9239-2dd43325fe3a service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:42.502178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84496595-6479-42bb-9239-2dd43325fe3a service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:42.502449 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84496595-6479-42bb-9239-2dd43325fe3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:42.511075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84496595-6479-42bb-9239-2dd43325fe3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:42.512697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84496595-6479-42bb-9239-2dd43325fe3a service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:42.520519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1185/2368] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:48:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:48:43.048323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.048683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.049072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.073038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:43.074285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:43.086782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1184/2369] 10.222.0.52 () {58 vars in 967 bytes} [Fri Oct 28 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:43.090338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.090788 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.091157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.100665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:43.113885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:43.114188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24c9e4-d3d0-445d-b9b6-00d8b276c3e1 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:43.115119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:43.119934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e60b3-bf4d-437a-af63-ca2712931944 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:43.120543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e60b3-bf4d-437a-af63-ca2712931944 service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.120817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e60b3-bf4d-437a-af63-ca2712931944 service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.121084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e60b3-bf4d-437a-af63-ca2712931944 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.124608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1186/2370] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:43.129885 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e60b3-bf4d-437a-af63-ca2712931944 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:43.132502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e60b3-bf4d-437a-af63-ca2712931944 service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:43.148906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:43.151779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1185/2371] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:48:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 28 09:48:43.162984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1a67a-b9c9-4eb3-848d-23cfdaa515ef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:43.170297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:43.170913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.171187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.171459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.180972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:43.182659 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:43.193368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1187/2372] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:48:43.426241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883861e-1677-491d-9dc3-9434d42cea25 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.426730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883861e-1677-491d-9dc3-9434d42cea25 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.427025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883861e-1677-491d-9dc3-9434d42cea25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.451185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4883861e-1677-491d-9dc3-9434d42cea25 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:43.452519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4883861e-1677-491d-9dc3-9434d42cea25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:43.460567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1186/2373] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:43.475257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:43.489157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369bc8ea-edea-42d3-9695-655d41a75532 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:43.494505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:43.495111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.495387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.495680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.504207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:43.505785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:43.513714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1188/2374] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:48:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:48:43.838892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.839178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.839452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.858811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:43.860010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:43.867159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1187/2375] 10.222.0.52 () {58 vars in 967 bytes} [Fri Oct 28 09:48:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:43.879220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:43.892348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07624c8-a450-4c81-a7f8-1586a4020010 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:43.897345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47be81-a507-415a-973c-f0bf8270853d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:43.898133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47be81-a507-415a-973c-f0bf8270853d service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:43.898412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47be81-a507-415a-973c-f0bf8270853d service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:43.898680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47be81-a507-415a-973c-f0bf8270853d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:43.907064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47be81-a507-415a-973c-f0bf8270853d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:43.908661 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47be81-a507-415a-973c-f0bf8270853d service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:43.916379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1189/2376] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:48:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:48:46.035362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:46.036771 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:46.050413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c4ada-224b-4120-a119-6bd99fedbdd6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:46.054835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:46.055489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.055791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.056063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] PATH_INFO: `/v3/users/c05718633def4ad6822728767580b810` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.057181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05718633def4ad6822728767580b810)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:46.059067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:46.097392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:46.106607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05718633def4ad6822728767580b810'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.114239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05718633def4ad6822728767580b810'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.121417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05718633def4ad6822728767580b810'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.122043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05718633def4ad6822728767580b810'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.136866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05718633def4ad6822728767580b810'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.143064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1188/2377] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:48:46 2022] DELETE /identity/v3/users/c05718633def4ad6822728767580b810 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:46.147436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.148873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.149762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.168693 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:46.169887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:46.208772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:46.227678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1190/2378] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:46.237808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:46.239251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:46.298076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab291-54a7-4e4b-aa7a-3732d6f6b6ed admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:46.302067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aecf28-9881-4378-8a8e-9958e93b71ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:46.302667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aecf28-9881-4378-8a8e-9958e93b71ed service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.302940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aecf28-9881-4378-8a8e-9958e93b71ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.303208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aecf28-9881-4378-8a8e-9958e93b71ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.319440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aecf28-9881-4378-8a8e-9958e93b71ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:46.321385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aecf28-9881-4378-8a8e-9958e93b71ed service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:46.331231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1189/2379] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:46.506090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:46.507501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:46.519646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e1f99-8684-470d-a63f-c5e30c3e9d7a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:46.524963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:46.525600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.525866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.526306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] PATH_INFO: `/v3/projects/d89c15d74a584be3b5e44f74ed50eea3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.527578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d89c15d74a584be3b5e44f74ed50eea3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:46.529657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:46.580197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd89c15d74a584be3b5e44f74ed50eea3', 'request_id': 'req-8354b694-eae5-40d1-947f-3b4cb9e101c1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.587802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd89c15d74a584be3b5e44f74ed50eea3', 'request_id': 'req-8354b694-eae5-40d1-947f-3b4cb9e101c1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.594312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd89c15d74a584be3b5e44f74ed50eea3', 'request_id': 'req-8354b694-eae5-40d1-947f-3b4cb9e101c1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.607555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] The token cache is being invalidate because project d89c15d74a584be3b5e44f74ed50eea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:48:46.607905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.608566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8354b694-eae5-40d1-947f-3b4cb9e101c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:46.610128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1191/2380] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:48:46 2022] DELETE /identity/v3/projects/d89c15d74a584be3b5e44f74ed50eea3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:46.615167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.615535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.615859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.636859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:46.638233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:46.714611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1190/2381] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:46.722207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.722614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.722960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.744288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:46.745500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:46.755275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1192/2382] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:46.759536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:46.769414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4eaaf5-5d45-4a37-9b96-ca1ccf51a7ec None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:46.774039 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:46.774640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.774910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.775216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.775844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:48:46.776142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:46.777708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:46.790147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1191/2383] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:48:46.794509 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:46.803603 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a0121-a77a-4a86-b2e1-aa147dd714cb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:46.808882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:46.809481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.809748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.810012 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.810657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:46.812639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:46.842893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1193/2384] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:48:46 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:48:46.855154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:46.871506 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf09ec1-9204-45c8-ae2d-9ceab3ab030f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:46.934470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:46.954584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:46.954864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:46.954967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:46.955195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:46.955854 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:46.958033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:47.006147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1192/2385] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:48:46 2022] POST /identity/v3/users => generated 299 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:48:47.010319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:47.021670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2905c-e6ea-4c3d-8b0a-19039bb2c172 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:47.026642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:47.027197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:47.027474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:47.027762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:47.028712 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:47.030588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:47.048236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1194/2386] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:48:47.049440 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:47.061485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4a071-f2e4-4fca-988e-a4a2c56c51c5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:47.068699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74751f2a-3868-4141-91fd-4a3507a7bf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:47.069612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751f2a-3868-4141-91fd-4a3507a7bf8e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:47.069986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751f2a-3868-4141-91fd-4a3507a7bf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:47.070402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751f2a-3868-4141-91fd-4a3507a7bf8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:47.071538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74751f2a-3868-4141-91fd-4a3507a7bf8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:47.073418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74751f2a-3868-4141-91fd-4a3507a7bf8e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:47.097443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1193/2387] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:48:47.102692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:47.117093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6967723-ad9c-4bd0-be00-dfed1ed8e801 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:47.137663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:47.138357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:47.138768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:47.138944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] PATH_INFO: `/v3/projects/9f82b7dee0864b30a312f1100875e6b8/users/5710127c76344bbeb91d8cb090e0307a/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:47.219707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f82b7dee0864b30a312f1100875e6b8, user_id=5710127c76344bbeb91d8cb090e0307a, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:47.222387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:47.248698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1195/2388] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:48:47 2022] PUT /identity/v3/projects/9f82b7dee0864b30a312f1100875e6b8/users/5710127c76344bbeb91d8cb090e0307a/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:47.253263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2db98-d3cf-45b7-ae4b-4c391522d4bc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:47.253551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2db98-d3cf-45b7-ae4b-4c391522d4bc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:47.253827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a2db98-d3cf-45b7-ae4b-4c391522d4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:47.294193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-44a2db98-d3cf-45b7-ae4b-4c391522d4bc None None] MFA Rules not processed for user `5710127c76344bbeb91d8cb090e0307a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:47.295541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-44a2db98-d3cf-45b7-ae4b-4c391522d4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:47.384619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1194/2389] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:47.395952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:47.397825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:47.461621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771f680-b0f3-4bab-8e07-178840ac7a59 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:47.467442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:47.468211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:47.468576 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:47.468871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:47.482618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:47.484367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:47.497956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1196/2390] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:50.420222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:50.421219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:50.421358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:50.441505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:50.442727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:50.517414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1195/2391] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:50.559464 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:50.561011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:50.571672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:50.647435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bfd9a-f14a-4e94-b855-d6ed07fdf529 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:50.654314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563cb2ea-b1a9-4a8a-86c5-79f47d738deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:50.654970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cb2ea-b1a9-4a8a-86c5-79f47d738deb service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:50.655242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cb2ea-b1a9-4a8a-86c5-79f47d738deb service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:50.655517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563cb2ea-b1a9-4a8a-86c5-79f47d738deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:50.665310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cb2ea-b1a9-4a8a-86c5-79f47d738deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:50.666956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563cb2ea-b1a9-4a8a-86c5-79f47d738deb service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:50.678322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1197/2392] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:52.654666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:52.665296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3898110a-8c05-4383-a0b0-7df48bd5579b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:52.670598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:52.671488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:52.671851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:52.672303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] PATH_INFO: `/v3/users/5710127c76344bbeb91d8cb090e0307a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:52.673641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5710127c76344bbeb91d8cb090e0307a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:52.675967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:52.728225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5710127c76344bbeb91d8cb090e0307a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:52.729958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:52.737486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5710127c76344bbeb91d8cb090e0307a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:52.746102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5710127c76344bbeb91d8cb090e0307a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:52.746789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5710127c76344bbeb91d8cb090e0307a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:52.759244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5710127c76344bbeb91d8cb090e0307a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:52.765083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1196/2393] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:48:52 2022] DELETE /identity/v3/users/5710127c76344bbeb91d8cb090e0307a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:52.769501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:52.770004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:52.770126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:52.788560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:52.790258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:52.847011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1198/2394] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:52.857587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:52.858969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:52.916023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95251665-343e-4fca-9cfd-0904f19d0d34 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:52.919666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce438b3-5c1a-44a5-a4da-a5ac77f9fc34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:52.920295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce438b3-5c1a-44a5-a4da-a5ac77f9fc34 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:52.920568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce438b3-5c1a-44a5-a4da-a5ac77f9fc34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:52.920836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce438b3-5c1a-44a5-a4da-a5ac77f9fc34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:52.929680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce438b3-5c1a-44a5-a4da-a5ac77f9fc34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:52.931675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce438b3-5c1a-44a5-a4da-a5ac77f9fc34 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:52.940990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1197/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:48:53.540536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:53.542055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:53.554300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8e4a28-2602-431c-b690-d44e1c0f324d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:48:53.559433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:48:53.560380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:53.560667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:53.560941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] PATH_INFO: `/v3/projects/9f82b7dee0864b30a312f1100875e6b8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:53.562260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f82b7dee0864b30a312f1100875e6b8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:48:53.564268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:48:53.631233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f82b7dee0864b30a312f1100875e6b8', 'request_id': 'req-584f2364-e741-4129-9364-6095979d5fd6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:53.641433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f82b7dee0864b30a312f1100875e6b8', 'request_id': 'req-584f2364-e741-4129-9364-6095979d5fd6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:53.647572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f82b7dee0864b30a312f1100875e6b8', 'request_id': 'req-584f2364-e741-4129-9364-6095979d5fd6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:53.669263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] The token cache is being invalidate because project 9f82b7dee0864b30a312f1100875e6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:48:53.670432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:53.670432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:48:53.672201 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1199/2396] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:48:53 2022] DELETE /identity/v3/projects/9f82b7dee0864b30a312f1100875e6b8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:48:59.797616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcaad4-c6d5-4693-b273-0ed272f54b75 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:48:59.798113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcaad4-c6d5-4693-b273-0ed272f54b75 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:48:59.798217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dcaad4-c6d5-4693-b273-0ed272f54b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:48:59.819869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-41dcaad4-c6d5-4693-b273-0ed272f54b75 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:48:59.821836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41dcaad4-c6d5-4693-b273-0ed272f54b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:59.903097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1198/2397] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:48:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:48:59.915995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:48:59.917390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:48:59.937578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:48:59.995129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2364-e741-4129-9364-6095979d5fd6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:00.000135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34888e-ea2d-4c92-b02e-3fac5a733e5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:00.000774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34888e-ea2d-4c92-b02e-3fac5a733e5a service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:00.001050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34888e-ea2d-4c92-b02e-3fac5a733e5a service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:00.001314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34888e-ea2d-4c92-b02e-3fac5a733e5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:00.010091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34888e-ea2d-4c92-b02e-3fac5a733e5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:00.011682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34888e-ea2d-4c92-b02e-3fac5a733e5a service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:00.019674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1200/2398] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:48:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:09.896015 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=63 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:10.016344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=25 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:14.924616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:14.925162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:14.925262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:14.943571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] MFA Rules not processed for user `36846c3821c14284893e25dd143d11e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:14.944814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:14.998621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1199/2399] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:49:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:15.470086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe04e8-cc2b-41e6-bdd1-ca20fa822b4d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:15.470627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe04e8-cc2b-41e6-bdd1-ca20fa822b4d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:15.470627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe04e8-cc2b-41e6-bdd1-ca20fa822b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:15.492133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4ffe04e8-cc2b-41e6-bdd1-ca20fa822b4d None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:15.493411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ffe04e8-cc2b-41e6-bdd1-ca20fa822b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:15.504898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1201/2400] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:49:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:15.519688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:15.533493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40724cd-675a-47f3-ad7e-16bb2192a405 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:15.545453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d28fcd-9080-4388-8224-0ae5ee1220d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:15.546275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28fcd-9080-4388-8224-0ae5ee1220d7 service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:15.546580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28fcd-9080-4388-8224-0ae5ee1220d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:15.546897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d28fcd-9080-4388-8224-0ae5ee1220d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:15.574921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28fcd-9080-4388-8224-0ae5ee1220d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:15.576613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d28fcd-9080-4388-8224-0ae5ee1220d7 service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:15.587436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1200/2401] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:49:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:18.020925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4158c5-daa5-48c4-b023-ea0c38d4d9da None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.021557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4158c5-daa5-48c4-b023-ea0c38d4d9da None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.021557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4158c5-daa5-48c4-b023-ea0c38d4d9da None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.022813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1202/2402] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:49:18.052098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4993f1-8001-4844-b18f-6734aedf140c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.053051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4993f1-8001-4844-b18f-6734aedf140c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.053526 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4993f1-8001-4844-b18f-6734aedf140c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.097020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5a4993f1-8001-4844-b18f-6734aedf140c None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:18.098544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5a4993f1-8001-4844-b18f-6734aedf140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:18.208181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1201/2403] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:49:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:18.223571 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4158c5-daa5-48c4-b023-ea0c38d4d9da None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:18.225558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1a4158c5-daa5-48c4-b023-ea0c38d4d9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:18.339677 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4158c5-daa5-48c4-b023-ea0c38d4d9da None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:18.344398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7282c-1e12-4b81-a301-4b4b90a9ebc7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:18.345435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7282c-1e12-4b81-a301-4b4b90a9ebc7 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.345871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7282c-1e12-4b81-a301-4b4b90a9ebc7 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.346286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7282c-1e12-4b81-a301-4b4b90a9ebc7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.368329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7282c-1e12-4b81-a301-4b4b90a9ebc7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:18.370394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7282c-1e12-4b81-a301-4b4b90a9ebc7 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:18.386997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1203/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:49:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:49:18.483664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93160b-74ea-405d-a54f-7deb5d904c63 None None] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.484419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93160b-74ea-405d-a54f-7deb5d904c63 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.484834 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93160b-74ea-405d-a54f-7deb5d904c63 None None] PATH_INFO: `/` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.486874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1202/2405] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:49:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:49:18.493730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5c296-0e62-4bb9-8c27-28748a0a7bc0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.494130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5c296-0e62-4bb9-8c27-28748a0a7bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.494990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5c296-0e62-4bb9-8c27-28748a0a7bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.534796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-66d5c296-0e62-4bb9-8c27-28748a0a7bc0 None None] MFA Rules not processed for user `b07b6ea0a9a7475fbcc8d6baf537cc4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:18.537954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-66d5c296-0e62-4bb9-8c27-28748a0a7bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:18.626978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1204/2406] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:49:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:18.644351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93160b-74ea-405d-a54f-7deb5d904c63 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:18.646598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-af93160b-74ea-405d-a54f-7deb5d904c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:18.663108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93160b-74ea-405d-a54f-7deb5d904c63 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:18.668926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc81236-2e84-4f36-8225-8e8cb9c828a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:18.669542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc81236-2e84-4f36-8225-8e8cb9c828a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.669818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc81236-2e84-4f36-8225-8e8cb9c828a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.670260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc81236-2e84-4f36-8225-8e8cb9c828a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.679662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc81236-2e84-4f36-8225-8e8cb9c828a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:18.681841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc81236-2e84-4f36-8225-8e8cb9c828a0 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:18.692824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1203/2407] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:49:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:49:18.881032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665313e-0144-498d-8419-5706782b197a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.881032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665313e-0144-498d-8419-5706782b197a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.881032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665313e-0144-498d-8419-5706782b197a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.882173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1205/2408] 10.222.0.52 () {54 vars in 921 bytes} [Fri Oct 28 09:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 28 09:49:18.888688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7cd1d-6915-49bf-8115-e31a6926d003 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.889267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7cd1d-6915-49bf-8115-e31a6926d003 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.889489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b7cd1d-6915-49bf-8115-e31a6926d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.911520 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-99b7cd1d-6915-49bf-8115-e31a6926d003 None None] MFA Rules not processed for user `9dc2bb9976f949548bc4743eb297b082`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:18.912780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-99b7cd1d-6915-49bf-8115-e31a6926d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:18.923124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1204/2409] 10.222.0.52 () {58 vars in 1018 bytes} [Fri Oct 28 09:49:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:18.934107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665313e-0144-498d-8419-5706782b197a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:18.950587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665313e-0144-498d-8419-5706782b197a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:18.957018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:18.957739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:18.958085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:18.958426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:18.986803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:18.988419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:19.002799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1206/2410] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:49:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:49:26.056390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42a80d-9a3d-4efe-9ce2-d519770377e9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:26.057026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42a80d-9a3d-4efe-9ce2-d519770377e9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:26.057026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42a80d-9a3d-4efe-9ce2-d519770377e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:26.078685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1c42a80d-9a3d-4efe-9ce2-d519770377e9 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:26.080894 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1c42a80d-9a3d-4efe-9ce2-d519770377e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:26.094023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1205/2411] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:49:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:26.117217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:26.141229 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb114-abac-4551-94e8-9da7a833cb27 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:26.154980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:26.155643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:26.155965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:26.156237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:26.166665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:26.168401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:26.177876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1207/2412] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:49:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:33.263528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:33.264213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:33.264213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:33.284055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:33.285265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:33.292637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1206/2413] 10.222.0.52 () {58 vars in 967 bytes} [Fri Oct 28 09:49:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:33.313220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:33.324504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5916eb-8ad1-4fd6-b318-86be830c0652 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:33.329731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:33.330436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:33.330775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:33.331120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:33.340495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:33.342188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:33.351011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1208/2414] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:49:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:35.474156 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:35.476164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:35.549768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:35.565644 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32473a58-daa2-4df3-9c2b-da5b2151360c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:35.571504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:35.572760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:35.573032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:35.573298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] PATH_INFO: `/v3/users/d10c1b55b5c04ba380ed8ffdaa462bc7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:35.574459 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10c1b55b5c04ba380ed8ffdaa462bc7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:35.576805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:35.643206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10c1b55b5c04ba380ed8ffdaa462bc7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:35.656031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10c1b55b5c04ba380ed8ffdaa462bc7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:35.664741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10c1b55b5c04ba380ed8ffdaa462bc7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:35.665473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10c1b55b5c04ba380ed8ffdaa462bc7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:35.677791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a31d239-182f-490b-88a4-4d27cea68cc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10c1b55b5c04ba380ed8ffdaa462bc7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:35.683721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1207/2415] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:49:35 2022] DELETE /identity/v3/users/d10c1b55b5c04ba380ed8ffdaa462bc7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:49:35.883240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:35.898669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfd6d-beac-49bc-8820-c045cf4f4c13 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:35.982899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:35.983861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:35.984134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:35.984404 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] PATH_INFO: `/v3/projects/2aa6bd99403b4421acb8bd714c9450c6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:35.985523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa6bd99403b4421acb8bd714c9450c6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:35.987561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:35.996892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:36.051328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa6bd99403b4421acb8bd714c9450c6', 'request_id': 'req-53117a70-5e3f-4281-853b-bb87fc425efd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:36.059577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa6bd99403b4421acb8bd714c9450c6', 'request_id': 'req-53117a70-5e3f-4281-853b-bb87fc425efd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:36.065645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa6bd99403b4421acb8bd714c9450c6', 'request_id': 'req-53117a70-5e3f-4281-853b-bb87fc425efd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:36.078097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] The token cache is being invalidate because project 2aa6bd99403b4421acb8bd714c9450c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:49:36.078383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:36.079046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:49:36.080557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1209/2416] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:49:35 2022] DELETE /identity/v3/projects/2aa6bd99403b4421acb8bd714c9450c6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:49:45.680231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:46.074823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:49:50.234892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3427-9967-449a-8ed0-49196511751d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:50.235438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3427-9967-449a-8ed0-49196511751d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:50.235438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d3427-9967-449a-8ed0-49196511751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:50.256107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-731d3427-9967-449a-8ed0-49196511751d None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:50.257490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-731d3427-9967-449a-8ed0-49196511751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:50.319299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1208/2417] 10.222.0.52 () {58 vars in 967 bytes} [Fri Oct 28 09:49:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:50.341692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:50.343125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:50.421113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53117a70-5e3f-4281-853b-bb87fc425efd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:50.426331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:50.426950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:50.427222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:50.427489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:50.436327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:50.437958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:50.446474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1210/2418] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:49:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:49:51.837323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ac9c4-e9d6-4cb5-ba04-eb0f9a5a305a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:51.837820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ac9c4-e9d6-4cb5-ba04-eb0f9a5a305a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:51.837938 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ac9c4-e9d6-4cb5-ba04-eb0f9a5a305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:51.859404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c42ac9c4-e9d6-4cb5-ba04-eb0f9a5a305a None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:51.860636 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c42ac9c4-e9d6-4cb5-ba04-eb0f9a5a305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:51.921360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1209/2419] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:51.942737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:51.944206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:52.023461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba37cb-23cc-4f19-82d5-c2866f252150 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:52.029830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:52.030450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:52.030728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:52.031001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:52.039865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:52.041487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:52.052162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1211/2420] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:49:56.149279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5755f-c26b-4142-982e-7aff3fb912bc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:56.149743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5755f-c26b-4142-982e-7aff3fb912bc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:56.149881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5755f-c26b-4142-982e-7aff3fb912bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:56.176646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cda5755f-c26b-4142-982e-7aff3fb912bc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:49:56.178032 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cda5755f-c26b-4142-982e-7aff3fb912bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:56.186488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1210/2421] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:49:56.211391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:49:56.212951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:49:56.228790 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f68bf-fb70-4aad-9c25-7e81b4659e88 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:49:56.235840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:49:56.236450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:49:56.237666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:49:56.237666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:49:56.246512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:49:56.248260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:49:56.258264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1212/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:00.302455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ed98-2c07-4761-90eb-100661acbff9 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:00.303062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ed98-2c07-4761-90eb-100661acbff9 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:00.303319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ed98-2c07-4761-90eb-100661acbff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:00.328619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-15f6ed98-2c07-4761-90eb-100661acbff9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:50:00.329853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-15f6ed98-2c07-4761-90eb-100661acbff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:00.337986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1211/2423] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:50:00.362885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:00.374925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818225-4564-4878-a833-91ea243f7dd7 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:00.380881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:00.381422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:00.381700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:00.381974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:00.391644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:00.393280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:00.403991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1213/2424] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:06.618549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee77b3-abdc-4c51-a4a2-94048a42bd0e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:06.619177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee77b3-abdc-4c51-a4a2-94048a42bd0e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:06.619177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee77b3-abdc-4c51-a4a2-94048a42bd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:06.639790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-7cee77b3-abdc-4c51-a4a2-94048a42bd0e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:50:06.640992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7cee77b3-abdc-4c51-a4a2-94048a42bd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:06.649211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1212/2425] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:50:06.686068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:06.697212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3ebdf-f429-4df3-933a-6ff9778905d2 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:06.703279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:06.703926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:06.704205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:06.704476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:06.719160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:06.722520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:06.731478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1214/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:10.821734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:10.822314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:10.822314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:10.842222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:50:10.843490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:10.851959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1213/2427] 10.222.0.58 () {58 vars in 1016 bytes} [Fri Oct 28 09:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:50:10.876385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:10.887460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02f653-5691-42b6-924c-492b3d6af93c service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:10.893115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:10.893702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:10.893978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:10.894250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:10.903257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:10.904842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:10.915257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1215/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:20.836999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=64 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:20.910716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:22.165544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:22.166955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:22.225168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb9f5e-0ef7-409f-93f3-9ba0594b951e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:22.228634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:22.229234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:22.229505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:22.229770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:22.232152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:22.245720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:22.247175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:22.258254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1214/2429] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:23.466162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:23.475552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466bfff-39c2-4360-bd2f-39799faa23c9 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:23.479289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:23.480052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:23.480418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:23.480882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:23.483127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:23.498177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:23.499649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:23.514138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1216/2430] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:32.253429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=26 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:33.506774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:45.786379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:45.787974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:45.801665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521aa21-5ae3-4e17-a258-d4edbcf8475d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:45.806768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:45.807527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:45.807793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:45.808070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] PATH_INFO: `/v3/users/36846c3821c14284893e25dd143d11e2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:45.809174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36846c3821c14284893e25dd143d11e2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:45.811735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:45.857058 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36846c3821c14284893e25dd143d11e2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:45.865347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36846c3821c14284893e25dd143d11e2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:45.873700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36846c3821c14284893e25dd143d11e2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:45.874369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36846c3821c14284893e25dd143d11e2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:45.886415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36846c3821c14284893e25dd143d11e2'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:45.892776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1215/2431] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:50:45 2022] DELETE /identity/v3/users/36846c3821c14284893e25dd143d11e2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:50:46.067063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:46.083431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f419be-e67d-4694-b878-f82da138ac3d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:46.141198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:46.141823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:46.142095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:46.142367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] PATH_INFO: `/v3/projects/23f1d6189cfd4437a13a0908119c7ecb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:46.143470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=23f1d6189cfd4437a13a0908119c7ecb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:46.145401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:46.193391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23f1d6189cfd4437a13a0908119c7ecb', 'request_id': 'req-0850a671-283c-40e0-9774-810710b8d98c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:46.200519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23f1d6189cfd4437a13a0908119c7ecb', 'request_id': 'req-0850a671-283c-40e0-9774-810710b8d98c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:46.206252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23f1d6189cfd4437a13a0908119c7ecb', 'request_id': 'req-0850a671-283c-40e0-9774-810710b8d98c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:46.217982 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] The token cache is being invalidate because project 23f1d6189cfd4437a13a0908119c7ecb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:50:46.218260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:46.219029 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0850a671-283c-40e0-9774-810710b8d98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:50:46.220604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1217/2432] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:50:46 2022] DELETE /identity/v3/projects/23f1d6189cfd4437a13a0908119c7ecb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:50:55.661665 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:55.663098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:55.721875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6f0b9-b2cb-42c3-a4d5-5679082a6ea3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:55.772487 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:55.773095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:55.773390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:55.773704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:55.775400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:55.785743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:55.787155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:55.795292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1216/2433] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:50:56.214500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:58.166451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:58.167020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:58.167020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:58.185793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] MFA Rules not processed for user `cfe6eb38e8df492daaef5755ddea7773`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:50:58.186973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:58.239220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1218/2434] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:50:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:50:58.589077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:50:58.594083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:58.637540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:50:58.685422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815e09-c2a8-484d-bdca-56a205d89309 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:50:58.692136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:50:58.692764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:50:58.693269 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:50:58.693385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:50:58.696204 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:50:58.759583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:50:58.761338 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1251ca-ea3e-4795-b458-238fdc9a37ef service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:50:58.771165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1217/2435] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:50:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:51:06.897492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:51:06.899133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:06.918285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012b9bf-9cee-4623-9a59-0ae6ba0f936d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:51:06.925884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:51:06.926925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:06.927491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:06.928087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:06.930260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:06.947013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:51:06.949015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:51:06.961495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1219/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:51:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:51:08.766473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=42 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:10.395164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:10.395786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:10.395786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:10.414775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] MFA Rules not processed for user `cdb5ce192c764fe4946542f5960ccd39`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:51:10.416007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:10.422694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1218/2437] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:51:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:51:10.734986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:51:10.737308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:10.748534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86e8d2-534d-40c9-8e3c-848f5aaff964 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:51:10.752114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:51:10.753280 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:10.753558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:10.753825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:10.755502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:10.764638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:51:10.766076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:51:10.773891 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1220/2438] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:51:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:51:11.097445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:51:11.098899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:11.181844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ef25d-664f-465b-bf53-727199b46d33 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:51:11.186800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:51:11.187404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:11.187679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:11.187978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:11.189729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:11.201537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:51:11.203088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:51:11.213422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1219/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:51:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:51:19.866444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:51:19.881650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54b902-d1b7-4716-95a2-c299c70c1049 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:51:19.887323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:51:19.887961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:19.888237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:19.888506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:19.890215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:19.904884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:51:19.906990 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e957c-cc03-48a3-9080-b6a1eb2e9aee service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:51:19.920204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1221/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:51:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:51:21.209202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=35 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:29.915867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=43 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:35.079454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:51:35.080924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:35.144741 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d51b1-1bcf-449b-95a6-2b8eda8f1831 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:51:35.149350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:51:35.149540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:35.149817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:35.150084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:35.151873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:35.211256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:51:35.212703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:51:35.221637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1220/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:51:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:51:40.913361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:40.913939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:40.913939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:40.933133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] MFA Rules not processed for user `8bee2d5f599642788c32e0d8b8338c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:51:40.934363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:40.941176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1222/2442] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:51:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:51:41.433000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:51:41.442794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89472d5-66ad-479b-9f51-0e032195b686 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:51:41.446379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:51:41.446972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:51:41.447246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:51:41.447513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:51:41.449192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:51:41.458493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:51:41.460096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:51:41.468381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1221/2443] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:51:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:51:50.928204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:51:51.464757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=54 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:09.029476 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:09.039392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf22fe-4cd9-47fa-87dc-ab6aeb4ceabc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:09.043014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:09.043647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:09.043962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:09.044234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:09.045968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:09.058152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:09.059624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:09.071108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1223/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:52:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:52:19.065477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:28.960050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:28.969301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb74a6a-3695-4943-b887-323b92c8939d service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:28.972957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:28.973553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:28.973824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:28.974093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:28.975812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:29.037334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:29.038753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:29.048279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1222/2445] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:52:32.061876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.076368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70eef0-f875-4f6f-b6f2-5282e5b0dcd4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.083005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.084135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.084736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.085215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.088142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:32.109766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.111486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.123641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1224/2446] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:52:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:52:32.161231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.162448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:32.177849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c9d97-c298-469b-8b3f-952d6d4524a9 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.183433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.184131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.184524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.184673 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] PATH_INFO: `/v3/users/cfe6eb38e8df492daaef5755ddea7773` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.185824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe6eb38e8df492daaef5755ddea7773)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.187836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.241041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe6eb38e8df492daaef5755ddea7773'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.248139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe6eb38e8df492daaef5755ddea7773'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.255041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe6eb38e8df492daaef5755ddea7773'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.255720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe6eb38e8df492daaef5755ddea7773'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.267694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b01eb750-c417-4e16-8317-e36b610d9370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe6eb38e8df492daaef5755ddea7773'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.273852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1223/2447] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:52:32 2022] DELETE /identity/v3/users/cfe6eb38e8df492daaef5755ddea7773 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:52:32.450101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.461747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a4cf5-4d53-4683-9861-098f5e4f923c service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.521376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.522009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.522289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.522561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] PATH_INFO: `/v3/projects/234f69f21d7a456a9ab6a86c4ac2ede2` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.523670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234f69f21d7a456a9ab6a86c4ac2ede2)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.525617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.571055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234f69f21d7a456a9ab6a86c4ac2ede2', 'request_id': 'req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.578528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234f69f21d7a456a9ab6a86c4ac2ede2', 'request_id': 'req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.584173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234f69f21d7a456a9ab6a86c4ac2ede2', 'request_id': 'req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.595332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] The token cache is being invalidate because project 234f69f21d7a456a9ab6a86c4ac2ede2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:52:32.595623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.596295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-70514a3a-57d8-41ff-9fcb-68d3a09e88c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:52:32.598036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1225/2448] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:52:32 2022] DELETE /identity/v3/projects/234f69f21d7a456a9ab6a86c4ac2ede2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:52:32.603382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.603667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.603968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.624470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:52:32.625615 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:32.684241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1224/2449] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:52:32.692131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.692411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.692689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.712818 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:52:32.713996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:32.721839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1226/2450] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:52:32.726243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.737296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965657d-ab37-433b-86dc-e3261700e76b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.742241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.742845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.743112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.743995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.744640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:52:32.744933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.746543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.748223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:32.761791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1225/2451] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:52:32.766150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.776259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba4598-abc0-4d36-88ae-461f15f72498 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.782027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.782635 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.782906 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.783172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.783923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.785848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.813719 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1227/2452] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:52:32 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:52:32.817906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.828689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21765b9-dcaa-4d25-95f4-b1e571ef2d22 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.886619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.887148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.887426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.887693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.888729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.890248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.910765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1226/2453] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:52:32 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:52:32.915191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.925492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648165d0-6e25-4d7e-b09e-10da5e1514c9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.930368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.931642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.931986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.932262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.932902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.934605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.944692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1228/2454] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:52:32.949257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.959136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe481d39-dbb8-4148-a180-ed840d5a376c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.963788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af417b0c-fcf1-4fb0-9d42-92706b241391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.964413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af417b0c-fcf1-4fb0-9d42-92706b241391 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.964691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af417b0c-fcf1-4fb0-9d42-92706b241391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.964961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af417b0c-fcf1-4fb0-9d42-92706b241391 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:32.965594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af417b0c-fcf1-4fb0-9d42-92706b241391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:32.967169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af417b0c-fcf1-4fb0-9d42-92706b241391 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:32.977611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1227/2455] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:52:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:52:32.981971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:32.992313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7cf48-10b7-47cb-9ef8-62752ad7dd6a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:32.997458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:32.998106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:32.998423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:32.998734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] PATH_INFO: `/v3/projects/68e60d7c624e49e1bb43188ae9d7dd2b/users/d38ce75423b5412d90c1158396978b97/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:33.013705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e60d7c624e49e1bb43188ae9d7dd2b, user_id=d38ce75423b5412d90c1158396978b97, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:33.016607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:33.028716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1229/2456] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:52:32 2022] PUT /identity/v3/projects/68e60d7c624e49e1bb43188ae9d7dd2b/users/d38ce75423b5412d90c1158396978b97/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:52:33.033957 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9aad7-03f3-44a2-a0a2-8ad32ae99a77 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:33.034237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9aad7-03f3-44a2-a0a2-8ad32ae99a77 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:33.034522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9aad7-03f3-44a2-a0a2-8ad32ae99a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:33.053435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6fb9aad7-03f3-44a2-a0a2-8ad32ae99a77 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:52:33.054637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6fb9aad7-03f3-44a2-a0a2-8ad32ae99a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:33.114786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1228/2457] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:52:33.124520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:33.126342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:33.141244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:33.185006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67683e7d-2542-48fa-bd84-f078c62feb85 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:33.188710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:33.189312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:33.189582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:33.189847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:33.199261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:33.200743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:33.209882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1230/2458] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:52:37.094843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bdaae-ef38-417c-87eb-e977b848daec None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:37.095310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bdaae-ef38-417c-87eb-e977b848daec None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:37.095427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bdaae-ef38-417c-87eb-e977b848daec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:37.124418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e56bdaae-ef38-417c-87eb-e977b848daec None None] MFA Rules not processed for user `d38ce75423b5412d90c1158396978b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:52:37.125664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e56bdaae-ef38-417c-87eb-e977b848daec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:37.201568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1229/2459] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:52:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:52:37.220133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:37.223166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:37.304860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052066e5-fc18-4c56-aa67-619c1eae88ae service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:37.311450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:37.312242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:37.312726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:37.313001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:37.322180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:37.323834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:37.333755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1231/2460] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:52:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:52:38.633286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfec58-3cd6-43ff-ad6c-65d0fd896810 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:38.633806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfec58-3cd6-43ff-ad6c-65d0fd896810 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:38.633941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfec58-3cd6-43ff-ad6c-65d0fd896810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:38.641141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:38.655098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-26dfec58-3cd6-43ff-ad6c-65d0fd896810 None None] MFA Rules not processed for user `d38ce75423b5412d90c1158396978b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:52:38.656474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-26dfec58-3cd6-43ff-ad6c-65d0fd896810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:38.663352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1230/2461] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:52:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:52:47.329706 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=54 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:48.650606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:52:52.449284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:52:52.451197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:52.475492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b9406-29b2-4693-a618-ecbd851fd4ad service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:52:52.485642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:52:52.486566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:52:52.487022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:52:52.487461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:52:52.491520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:52:52.512150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:52:52.514995 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:52:52.533730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1232/2462] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:52:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:53:02.525861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:05.686646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694da3-0fba-41d7-acee-fd9d43ddb3d4 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:05.687147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694da3-0fba-41d7-acee-fd9d43ddb3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:05.687268 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694da3-0fba-41d7-acee-fd9d43ddb3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:05.711117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-05694da3-0fba-41d7-acee-fd9d43ddb3d4 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:53:05.712359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-05694da3-0fba-41d7-acee-fd9d43ddb3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:05.767756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1231/2463] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:53:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:53:05.815233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:53:05.816670 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:05.885624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde7dc5-e121-4ede-a1d3-1ae799163403 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:53:05.891456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:53:05.892100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:05.892375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:05.892752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:05.901235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:53:05.902835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:53:05.910928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1233/2464] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:53:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:53:15.760413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=25 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:15.907589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:22.339056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:22.339535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:22.339630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:22.378340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:53:22.379531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:22.412019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1232/2465] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:53:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:53:22.424446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:53:22.464005 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd927b1-6936-4b99-af51-66052e680c03 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:53:22.469548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a58b4-3869-4f82-838e-54f7ebe76304 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:53:22.470141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a58b4-3869-4f82-838e-54f7ebe76304 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:22.470418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a58b4-3869-4f82-838e-54f7ebe76304 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:22.470694 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a58b4-3869-4f82-838e-54f7ebe76304 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:22.478741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a58b4-3869-4f82-838e-54f7ebe76304 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:53:22.480363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a58b4-3869-4f82-838e-54f7ebe76304 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:53:22.488981 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1234/2466] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:53:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:53:32.410182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=23 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:32.486157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=20 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:38.134887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:53:38.136345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:38.230838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53919a83-9c2d-43db-a589-b58e0d45f15c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:53:38.236424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:53:38.237033 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:38.237310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:38.237661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:38.239342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:38.251225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:53:38.252843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:53:38.262980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1233/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:53:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:53:42.327561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:42.328077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:42.328196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:42.348306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:53:42.349507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:42.356793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1235/2468] 10.222.0.52 () {58 vars in 967 bytes} [Fri Oct 28 09:53:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:53:42.369332 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:53:42.381031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc647a-c4d9-46eb-8c61-12ca1367c730 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:53:42.385856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03305620-2153-4530-9192-0a74297a17ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:53:42.386463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305620-2153-4530-9192-0a74297a17ad service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:42.386738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305620-2153-4530-9192-0a74297a17ad service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:42.387006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305620-2153-4530-9192-0a74297a17ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:42.396485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03305620-2153-4530-9192-0a74297a17ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:53:42.397991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03305620-2153-4530-9192-0a74297a17ad service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:53:42.408256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1234/2469] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:53:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:53:52.341473 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:52.402953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=47 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:53:58.540297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:53:58.541631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:58.581206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19ed98-09c6-4e93-99fb-ac6cba60aac7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:53:58.585035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:53:58.585651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:53:58.585931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:53:58.586204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:53:58.588032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:53:58.627524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:53:58.629059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:53:58.651303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1236/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:02.020191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03305620-2153-4530-9192-0a74297a17ad service swift] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:02.031908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03305620-2153-4530-9192-0a74297a17ad service swift] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:02.037076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:02.037686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.037963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.038237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.040092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:02.052493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:02.054109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:02.067687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1235/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:02.080337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:02.090261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:02.111812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e6a1a-1897-4607-abc7-b395ec429b84 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:02.116887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:02.117519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.117795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.118091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] PATH_INFO: `/v3/users/8bee2d5f599642788c32e0d8b8338c19` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.119203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bee2d5f599642788c32e0d8b8338c19)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:02.121173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:02.171830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bee2d5f599642788c32e0d8b8338c19'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.179307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bee2d5f599642788c32e0d8b8338c19'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.191988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bee2d5f599642788c32e0d8b8338c19'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.192723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bee2d5f599642788c32e0d8b8338c19'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.200826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-06dc7639-f1a6-4b33-bdd9-84169ad2b411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bee2d5f599642788c32e0d8b8338c19'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.209136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1237/2472] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:02 2022] DELETE /identity/v3/users/8bee2d5f599642788c32e0d8b8338c19 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:02.398428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:02.416376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015d3cf-281b-4c27-bf5a-fb1df0034691 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:02.486226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:02.486782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.487057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.487333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] PATH_INFO: `/v3/projects/ae5e7d531c0f4a33a869309af965e3e5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.488480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae5e7d531c0f4a33a869309af965e3e5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:02.490378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:02.536539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae5e7d531c0f4a33a869309af965e3e5', 'request_id': 'req-7150ced0-31e8-49a3-b940-87be91068bdc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.542082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae5e7d531c0f4a33a869309af965e3e5', 'request_id': 'req-7150ced0-31e8-49a3-b940-87be91068bdc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.548899 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae5e7d531c0f4a33a869309af965e3e5', 'request_id': 'req-7150ced0-31e8-49a3-b940-87be91068bdc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.561733 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] The token cache is being invalidate because project ae5e7d531c0f4a33a869309af965e3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:02.562020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.562660 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7150ced0-31e8-49a3-b940-87be91068bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:02.564162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1236/2473] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:02 2022] DELETE /identity/v3/projects/ae5e7d531c0f4a33a869309af965e3e5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:02.569168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.569460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.569737 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.630728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:02.631954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:02.664858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:02.822638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1238/2474] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:02.831301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.831485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.831810 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.851859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:02.853100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:02.861698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1237/2475] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:02.865729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:02.875581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68b648-7ca9-4282-93f2-9e06a0388689 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:02.880197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:02.880793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.881061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.881322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.881961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:02.882253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:02.884013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:02.896873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1239/2476] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:54:02.900884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:02.911895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a001dc-b4bc-4bdb-bafc-cf4e3b113bd6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:02.916518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:02.917113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:02.917406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:02.917672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:02.918423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:02.921263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:02.948027 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1238/2477] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:02 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:02.952293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:02.961953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8a59c-2920-426b-bcd7-ff8852e827a0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:03.026945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:03.027451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.027720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.028019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.028652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:03.031058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:03.053887 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1240/2478] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:02 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:03.058475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:03.069271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fddffe-ba74-44d7-9855-9f58e47c7c69 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:03.074021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:03.074632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.074910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.075185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.075867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:03.077967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:03.088397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1239/2479] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:03.093314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:03.103663 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8368cf0-ecd3-41b5-b1eb-ff041343d0ca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:03.108420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde76866-39fa-4f59-bc4b-8f37aae8ae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:03.109212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde76866-39fa-4f59-bc4b-8f37aae8ae9c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.109343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde76866-39fa-4f59-bc4b-8f37aae8ae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.109574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde76866-39fa-4f59-bc4b-8f37aae8ae9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.110223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde76866-39fa-4f59-bc4b-8f37aae8ae9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:03.111937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde76866-39fa-4f59-bc4b-8f37aae8ae9c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:03.123434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1241/2480] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:03.127701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:03.137994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11708c-0b9d-4e0d-abcd-0ef7f6a43972 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:03.143255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:03.143956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.144231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.144503 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] PATH_INFO: `/v3/projects/af86914d649d4cd88a7366c0693ab438/users/27d120bc73eb4bbea72153707d6e5caa/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.160307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af86914d649d4cd88a7366c0693ab438, user_id=27d120bc73eb4bbea72153707d6e5caa, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:03.162963 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:03.177586 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1240/2481] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:03 2022] PUT /identity/v3/projects/af86914d649d4cd88a7366c0693ab438/users/27d120bc73eb4bbea72153707d6e5caa/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:03.189024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.189313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.189582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.219657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] MFA Rules not processed for user `27d120bc73eb4bbea72153707d6e5caa`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:03.220904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:03.287457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1242/2482] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:03.299613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:03.301360 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:03.319887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:03.405755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6168e-f77b-431e-8ef3-b1a4fc8b063e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:03.411428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04d450c-cc9e-43b8-8966-7f8f37c3c746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:03.412168 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d450c-cc9e-43b8-8966-7f8f37c3c746 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.412454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d450c-cc9e-43b8-8966-7f8f37c3c746 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.412724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04d450c-cc9e-43b8-8966-7f8f37c3c746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.421068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d450c-cc9e-43b8-8966-7f8f37c3c746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:03.422754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04d450c-cc9e-43b8-8966-7f8f37c3c746 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:03.431919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1241/2483] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:03.668557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:03.707345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:03.732440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daf78a-50bf-4a1f-ba63-6c4287eb43b3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:03.744303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:03.744956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.745228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.745496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] PATH_INFO: `/v3/users/27d120bc73eb4bbea72153707d6e5caa` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.746608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d120bc73eb4bbea72153707d6e5caa)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:03.749068 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:03.799951 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d120bc73eb4bbea72153707d6e5caa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:03.809037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d120bc73eb4bbea72153707d6e5caa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:03.821875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d120bc73eb4bbea72153707d6e5caa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:03.822554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d120bc73eb4bbea72153707d6e5caa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:03.829665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d120bc73eb4bbea72153707d6e5caa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:03.835828 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1243/2484] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:03 2022] DELETE /identity/v3/users/27d120bc73eb4bbea72153707d6e5caa => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:03.841445 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:03.841732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:03.842010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:03.864641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:03.865873 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:03.927680 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1242/2485] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:03.938258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:03.939654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:03.996216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec1237-58e6-4aa0-adae-dd75938379e0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.000529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50f727-8842-48ab-b8fc-126c723109c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.001158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50f727-8842-48ab-b8fc-126c723109c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.001429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50f727-8842-48ab-b8fc-126c723109c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.001696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50f727-8842-48ab-b8fc-126c723109c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.010516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50f727-8842-48ab-b8fc-126c723109c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.011979 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50f727-8842-48ab-b8fc-126c723109c9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.021373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1244/2486] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:04.287432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.288913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:04.303124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6bec5-836a-4de5-a29a-46f951e2e011 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.307718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.308921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.309410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.309679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] PATH_INFO: `/v3/projects/af86914d649d4cd88a7366c0693ab438` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.310791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af86914d649d4cd88a7366c0693ab438)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.312740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.359569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af86914d649d4cd88a7366c0693ab438', 'request_id': 'req-2b8d0d33-2871-4772-b11b-a6a923276ae9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:04.365093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af86914d649d4cd88a7366c0693ab438', 'request_id': 'req-2b8d0d33-2871-4772-b11b-a6a923276ae9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:04.371346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af86914d649d4cd88a7366c0693ab438', 'request_id': 'req-2b8d0d33-2871-4772-b11b-a6a923276ae9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:04.386215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] The token cache is being invalidate because project af86914d649d4cd88a7366c0693ab438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:04.386629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:04.387428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-2b8d0d33-2871-4772-b11b-a6a923276ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:04.389063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1243/2487] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:04 2022] DELETE /identity/v3/projects/af86914d649d4cd88a7366c0693ab438 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:04.394243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.394736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.394841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.414674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:04.415928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:04.464835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:04.475860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1245/2488] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:54:04.483992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.484274 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.484567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.492544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:04.506666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:04.507932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:04.515919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1244/2489] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:04.523037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.535023 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba7933-0ec3-41dd-ac21-42cf020aff48 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.542421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.543048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.543374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.543724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.544410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:04.544711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.547000 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.558949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1246/2490] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:54:04.563878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.573372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f908c-723f-40ab-ae70-d0e2f4af88ba None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.578552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.579146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.579421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.579835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.580504 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.582384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.612403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1245/2491] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:04 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:04.616978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.626713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0ef2-f0ef-40af-913d-87d49261def3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.682944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.683565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.683944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.684167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.684843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.686796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.709415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1247/2492] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:04 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:04.713980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.725167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc6014-8857-4516-ba34-3ce671efcfa8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.729787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.730392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.730664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.730931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.731564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.733224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.743898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1246/2493] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:04.748707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.760933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd253bf-35a4-4fa5-933d-a3971225659b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.766375 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bbeedf-673c-4118-8287-7f95902b39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.766987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbeedf-673c-4118-8287-7f95902b39fc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.767263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbeedf-673c-4118-8287-7f95902b39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.767531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bbeedf-673c-4118-8287-7f95902b39fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.768205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbeedf-673c-4118-8287-7f95902b39fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.770017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bbeedf-673c-4118-8287-7f95902b39fc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.781701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1248/2494] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:04.786043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:04.796223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd749b1-f010-47d9-bab8-29cdd755c0c0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:04.800901 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:04.801547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.801824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.802097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] PATH_INFO: `/v3/projects/9181206f1d974f079015414293d8ccc5/users/117ed653ea5344aab3e571d473570f91/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.818221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9181206f1d974f079015414293d8ccc5, user_id=117ed653ea5344aab3e571d473570f91, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:04.820988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:04.834967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1247/2495] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:04 2022] PUT /identity/v3/projects/9181206f1d974f079015414293d8ccc5/users/117ed653ea5344aab3e571d473570f91/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:04.847137 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:04.847413 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:04.847684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:04.877666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] MFA Rules not processed for user `117ed653ea5344aab3e571d473570f91`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:04.878801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:04.945644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1249/2496] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:04.950366 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.015203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d9ae6-c9db-48df-93fd-b40c32c77261 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.020466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.021079 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.021354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.021624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.022262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.024213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.052192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1248/2497] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:04 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:05.056599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.070180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a703a4-146b-4d8d-b9c1-eb1f9b42f837 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.118909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:05.129597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.130116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.130389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.130658 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.131290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.133400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.157412 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1250/2498] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:05 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:05.161563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.172960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698eed73-14bc-4072-955f-9a469799b2e4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.178141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.178796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.179095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.179344 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.180026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.181643 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.193567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1249/2499] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:05.197651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.208691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03280ff-82a0-4999-84d4-830d711828cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.213344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.214001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.214278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.214553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] PATH_INFO: `/v3/projects/1cbeaaff967b41e9b301eb82f8e81fe2/users/d2b39b9f512d4f3fb44850db49c4425b/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.229858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbeaaff967b41e9b301eb82f8e81fe2, user_id=d2b39b9f512d4f3fb44850db49c4425b, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.232556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.246088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1251/2500] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:05 2022] PUT /identity/v3/projects/1cbeaaff967b41e9b301eb82f8e81fe2/users/d2b39b9f512d4f3fb44850db49c4425b/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:05.250626 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.260821 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba884b9-2431-41ca-97a8-d6e2eeee2628 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.318100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075e88d-07ca-4808-a261-865f10b17d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.318790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075e88d-07ca-4808-a261-865f10b17d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.318971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075e88d-07ca-4808-a261-865f10b17d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.319259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075e88d-07ca-4808-a261-865f10b17d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.319909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075e88d-07ca-4808-a261-865f10b17d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.321492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075e88d-07ca-4808-a261-865f10b17d01 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.331718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1250/2501] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:05.336014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.347124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98791cdf-e0be-4027-b58c-204c3da1373f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.351667 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.352342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.352619 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.352890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] PATH_INFO: `/v3/projects/1cbeaaff967b41e9b301eb82f8e81fe2/users/d2b39b9f512d4f3fb44850db49c4425b/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.354777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbeaaff967b41e9b301eb82f8e81fe2, user_id=d2b39b9f512d4f3fb44850db49c4425b, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.357462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.368997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1252/2502] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:05 2022] PUT /identity/v3/projects/1cbeaaff967b41e9b301eb82f8e81fe2/users/d2b39b9f512d4f3fb44850db49c4425b/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:05.381403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.381607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.381877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.401874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:05.413073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] MFA Rules not processed for user `d2b39b9f512d4f3fb44850db49c4425b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:05.414401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:05.483656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1251/2503] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:05.494843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.498236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:05.577761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed3431-c358-4164-a46d-6b4819963cb8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.581487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.582088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.582359 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.582626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.591666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.593196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.602805 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1253/2504] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:05.653393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:05.654783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:05.666808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbdc56-b54d-4df7-afb0-9b7ec3028f55 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:05.670964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2e397-0d68-4a67-9fe8-776e763b2bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:05.671573 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2e397-0d68-4a67-9fe8-776e763b2bfb service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:05.671877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2e397-0d68-4a67-9fe8-776e763b2bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:05.672143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2e397-0d68-4a67-9fe8-776e763b2bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:05.731718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2e397-0d68-4a67-9fe8-776e763b2bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:05.733809 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2e397-0d68-4a67-9fe8-776e763b2bfb service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:05.743736 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1252/2505] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:09.113392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:09.180767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef1491-8032-421c-a033-3316aabdd02d service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:09.185263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:09.185942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:09.186721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:09.186984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] PATH_INFO: `/v3/users/117ed653ea5344aab3e571d473570f91` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:09.188112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=117ed653ea5344aab3e571d473570f91)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:09.190022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:09.225581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:09.241637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '117ed653ea5344aab3e571d473570f91'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.249561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '117ed653ea5344aab3e571d473570f91'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.261900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '117ed653ea5344aab3e571d473570f91'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.262640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '117ed653ea5344aab3e571d473570f91'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.272812 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '117ed653ea5344aab3e571d473570f91'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.279673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1254/2506] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:09 2022] DELETE /identity/v3/users/117ed653ea5344aab3e571d473570f91 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:09.284309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:09.284827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:09.284827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:09.306734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:09.309094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:09.388460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1253/2507] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:09.399284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:09.400833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:09.460194 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89942ed-4c83-4c24-9622-0900cdfeffe4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:09.463869 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:09.464501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:09.464769 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:09.465032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:09.475092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:09.476565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:09.487225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1255/2508] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:09.779044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:09.784008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:09.805081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226e02f-285e-45c1-a741-3e2d5c3cf416 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:09.812743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:09.813628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:09.813974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:09.814313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] PATH_INFO: `/v3/projects/9181206f1d974f079015414293d8ccc5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:09.815804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9181206f1d974f079015414293d8ccc5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:09.817539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:09.856053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:09.874832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9181206f1d974f079015414293d8ccc5', 'request_id': 'req-271a87f6-e29a-4608-ae29-45ddf5f010bc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.881028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9181206f1d974f079015414293d8ccc5', 'request_id': 'req-271a87f6-e29a-4608-ae29-45ddf5f010bc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.887590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9181206f1d974f079015414293d8ccc5', 'request_id': 'req-271a87f6-e29a-4608-ae29-45ddf5f010bc'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.902238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] The token cache is being invalidate because project 9181206f1d974f079015414293d8ccc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:09.902511 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.903064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:09.905444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1254/2509] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:09 2022] DELETE /identity/v3/projects/9181206f1d974f079015414293d8ccc5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:09.909793 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:09.920541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a42cb-4750-4890-873c-53c139aa9b49 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:09.981547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:09.982184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:09.982453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:09.982728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] PATH_INFO: `/v3/users/d2b39b9f512d4f3fb44850db49c4425b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:09.983863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b39b9f512d4f3fb44850db49c4425b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:09.985825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.045039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b39b9f512d4f3fb44850db49c4425b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.049239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:10.054401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b39b9f512d4f3fb44850db49c4425b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.069837 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b39b9f512d4f3fb44850db49c4425b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.070600 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b39b9f512d4f3fb44850db49c4425b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.078587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0f97f271-95bf-45ae-b90f-c526c85ebd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b39b9f512d4f3fb44850db49c4425b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.085871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1256/2510] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:09 2022] DELETE /identity/v3/users/d2b39b9f512d4f3fb44850db49c4425b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:10.382568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:10.384145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:10.399780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a87f6-e29a-4608-ae29-45ddf5f010bc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:10.464650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:10.465181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.465454 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.465734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] PATH_INFO: `/v3/projects/1cbeaaff967b41e9b301eb82f8e81fe2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.466868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbeaaff967b41e9b301eb82f8e81fe2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:10.468847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.518053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbeaaff967b41e9b301eb82f8e81fe2', 'request_id': 'req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.523774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbeaaff967b41e9b301eb82f8e81fe2', 'request_id': 'req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.530059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbeaaff967b41e9b301eb82f8e81fe2', 'request_id': 'req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.542569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] The token cache is being invalidate because project 1cbeaaff967b41e9b301eb82f8e81fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:10.542850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.543483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c0b64c70-8f90-4a9d-bfcf-d2f515bde9b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:10.544978 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1255/2511] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:10 2022] DELETE /identity/v3/projects/1cbeaaff967b41e9b301eb82f8e81fe2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:10.550910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.551409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.551505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.577357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:10.578574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:10.637225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1257/2512] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:10.645041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.645318 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.645585 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.668194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:10.669497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:10.680931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1256/2513] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:10.685411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:10.697013 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ace1a-900f-4c4b-b2bb-8e7e83d3aff8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:10.701988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:10.702622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.702897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.703165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.703862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:10.704166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:10.705864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.718909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1258/2514] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:54:10.724757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:10.737337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a33de9-b214-44bb-8dbc-0c2119f49487 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:10.742580 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:10.743209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.743517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.743843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.744574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:10.746513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.787911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1257/2515] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:10 2022] POST /identity/v3/projects => generated 401 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:10.790596 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:10.805782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be60ffb-1105-48f3-9000-660f69b3b602 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:10.878218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:10.878949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.879282 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.879548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.880257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:10.882167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.908733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1259/2516] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:10 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:10.913152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:10.925740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4317-748e-48ff-be6e-1c13e6f6a79b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:10.930657 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:10.931303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.931700 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.931867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.932515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:10.934155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.945514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1258/2517] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:10.949962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:10.965031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f192e37-f5e5-401f-ade5-044a908afec6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:10.975518 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc79941-9863-4f70-a28a-8f95e41d0c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:10.976917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc79941-9863-4f70-a28a-8f95e41d0c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:10.977492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc79941-9863-4f70-a28a-8f95e41d0c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:10.977970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc79941-9863-4f70-a28a-8f95e41d0c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:10.979165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc79941-9863-4f70-a28a-8f95e41d0c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:10.984315 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc79941-9863-4f70-a28a-8f95e41d0c0d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:10.998408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1260/2518] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:11.003246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.014819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b78e2-84ea-4140-9f3d-c12b4525b86c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.019570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.020328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.020604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.020877 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] PATH_INFO: `/v3/projects/43294dcb868b4fbfb01b1846e326a66d/users/1ccd983ea40a478ca63e9680c058259c/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.024249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:11.040781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43294dcb868b4fbfb01b1846e326a66d, user_id=1ccd983ea40a478ca63e9680c058259c, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.044347 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.067895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1259/2519] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:11 2022] PUT /identity/v3/projects/43294dcb868b4fbfb01b1846e326a66d/users/1ccd983ea40a478ca63e9680c058259c/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:11.076299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.076578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.076960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.109893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] MFA Rules not processed for user `1ccd983ea40a478ca63e9680c058259c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:11.111106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:11.141130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:11.191402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1261/2520] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:11.196135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.269517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ddebec-db37-4ef0-b592-b97411c9a997 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.274110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.274722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.274998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.275263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.275927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.277816 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.309800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1260/2521] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:11 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:11.314597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.326098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80602802-5e16-46cc-a91a-b6e912f989d8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.388052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.388675 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.389443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.389795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.390437 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.392580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.417075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1262/2522] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:11 2022] POST /identity/v3/users => generated 306 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:11.421867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.433166 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b0fd3c-980f-4c3a-8347-590162190a89 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.439009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.439710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.440135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.440448 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.441234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.443169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.455988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1261/2523] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:11.460242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.471708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8f418-be17-4127-a798-bcd6a04c32aa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.476846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.477525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.477801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.478076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] PATH_INFO: `/v3/projects/2a230cac62c54799bd5c4d7549390963/users/4a0d10a0d12d4ac183a36c2471eb2727/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.494740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a230cac62c54799bd5c4d7549390963, user_id=4a0d10a0d12d4ac183a36c2471eb2727, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.497463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.512131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1263/2524] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:11 2022] PUT /identity/v3/projects/2a230cac62c54799bd5c4d7549390963/users/4a0d10a0d12d4ac183a36c2471eb2727/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:11.516705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.527010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0f8c5-61a2-4f31-adb7-c05ddb9a2aec admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.585755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98404d2-eef8-4267-a458-0bad968bb7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.586372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98404d2-eef8-4267-a458-0bad968bb7ca admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.586650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98404d2-eef8-4267-a458-0bad968bb7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.586922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98404d2-eef8-4267-a458-0bad968bb7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.587566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98404d2-eef8-4267-a458-0bad968bb7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.589211 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98404d2-eef8-4267-a458-0bad968bb7ca admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.600683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1262/2525] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:11.607133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.618204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffea15b-ee1f-4892-881b-6a90e66c03df admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.622925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.623578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.623894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.624163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] PATH_INFO: `/v3/projects/2a230cac62c54799bd5c4d7549390963/users/4a0d10a0d12d4ac183a36c2471eb2727/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.626070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a230cac62c54799bd5c4d7549390963, user_id=4a0d10a0d12d4ac183a36c2471eb2727, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.628755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.640300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1264/2526] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:11 2022] PUT /identity/v3/projects/2a230cac62c54799bd5c4d7549390963/users/4a0d10a0d12d4ac183a36c2471eb2727/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:11.650756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.651041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.651319 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.681364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] MFA Rules not processed for user `4a0d10a0d12d4ac183a36c2471eb2727`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:11.682595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:11.727622 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:11.757283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1263/2527] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:54:11.773292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.774971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:11.840318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4498821-9de0-4ce9-9111-3c117a143cc1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.844073 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.844695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.844969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.845279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:11.855124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:11.856626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:11.867515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1265/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:11 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:11.948718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:11.959923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b429fa-4cbc-4a4d-b599-65fa668e03c2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:11.963401 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139879d9-1a23-4ce6-a329-61a392844cdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:11.964035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139879d9-1a23-4ce6-a329-61a392844cdc service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:11.964313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139879d9-1a23-4ce6-a329-61a392844cdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:11.964577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139879d9-1a23-4ce6-a329-61a392844cdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:12.026624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139879d9-1a23-4ce6-a329-61a392844cdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:12.028150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139879d9-1a23-4ce6-a329-61a392844cdc service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:12.037518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1264/2529] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:11 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:16.206109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:16.229449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca577bad-7e97-4ace-9a7e-71849b9661b0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:16.229990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca577bad-7e97-4ace-9a7e-71849b9661b0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:16.229990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca577bad-7e97-4ace-9a7e-71849b9661b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:16.230434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:16.250299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ca577bad-7e97-4ace-9a7e-71849b9661b0 None None] MFA Rules not processed for user `d38ce75423b5412d90c1158396978b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:16.251583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ca577bad-7e97-4ace-9a7e-71849b9661b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:16.275259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977629e-bc63-44a9-b620-930732ea1652 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:16.280249 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:16.281494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:16.281711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:16.281984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] PATH_INFO: `/v3/users/1ccd983ea40a478ca63e9680c058259c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:16.283170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ccd983ea40a478ca63e9680c058259c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:16.285182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:16.306073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1265/2530] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:54:16.334022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ccd983ea40a478ca63e9680c058259c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:16.342163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ccd983ea40a478ca63e9680c058259c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:16.355096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ccd983ea40a478ca63e9680c058259c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:16.355802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ccd983ea40a478ca63e9680c058259c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:16.365016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ccd983ea40a478ca63e9680c058259c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:16.371173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1266/2531] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:16 2022] DELETE /identity/v3/users/1ccd983ea40a478ca63e9680c058259c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:16.375685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:16.376064 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:16.376403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:16.395944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:16.397165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:16.458826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1266/2532] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:16.469223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:16.470734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:16.535118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d557a64-3dce-4122-9ae8-c8ff122e7247 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:16.539395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:16.540177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:16.540541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:16.540803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:16.551264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:16.553253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:16.563269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1267/2533] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:54:16.919842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:16.921781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:16.938897 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae659c8-88de-4624-9d1a-e47386f64774 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:16.943604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:16.944282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:16.944578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:16.944853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] PATH_INFO: `/v3/projects/43294dcb868b4fbfb01b1846e326a66d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:16.945977 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43294dcb868b4fbfb01b1846e326a66d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:16.947998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:16.973789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:17.028513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43294dcb868b4fbfb01b1846e326a66d', 'request_id': 'req-fcd15e9c-8f46-46af-99d8-898f5214cbf6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.035604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43294dcb868b4fbfb01b1846e326a66d', 'request_id': 'req-fcd15e9c-8f46-46af-99d8-898f5214cbf6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.042399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43294dcb868b4fbfb01b1846e326a66d', 'request_id': 'req-fcd15e9c-8f46-46af-99d8-898f5214cbf6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.057071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] The token cache is being invalidate because project 43294dcb868b4fbfb01b1846e326a66d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:17.057625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.058342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.060097 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1267/2534] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:16 2022] DELETE /identity/v3/projects/43294dcb868b4fbfb01b1846e326a66d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:17.066167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:17.076644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ed776-3746-4381-91ae-d223824c8238 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:17.121519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:17.146852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:17.147947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.147947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.148176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] PATH_INFO: `/v3/users/4a0d10a0d12d4ac183a36c2471eb2727` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.149305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0d10a0d12d4ac183a36c2471eb2727)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:17.151233 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:17.197049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0d10a0d12d4ac183a36c2471eb2727'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.205314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0d10a0d12d4ac183a36c2471eb2727'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.221067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0d10a0d12d4ac183a36c2471eb2727'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.221848 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0d10a0d12d4ac183a36c2471eb2727'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.230189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-dd2ff199-2630-4226-9097-5cd435559d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0d10a0d12d4ac183a36c2471eb2727'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.237060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1268/2535] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:17 2022] DELETE /identity/v3/users/4a0d10a0d12d4ac183a36c2471eb2727 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:17.520444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:17.522197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:17.537198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd15e9c-8f46-46af-99d8-898f5214cbf6 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:17.597314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:17.597881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.598162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.598439 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] PATH_INFO: `/v3/projects/2a230cac62c54799bd5c4d7549390963` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.599561 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a230cac62c54799bd5c4d7549390963)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:17.601492 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:17.651107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a230cac62c54799bd5c4d7549390963', 'request_id': 'req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.657278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a230cac62c54799bd5c4d7549390963', 'request_id': 'req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.664656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a230cac62c54799bd5c4d7549390963', 'request_id': 'req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.678466 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] The token cache is being invalidate because project 2a230cac62c54799bd5c4d7549390963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:17.678779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.679421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-587934fc-f3d2-4254-ab77-2a4cd2f0a3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:17.680967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1268/2536] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:17 2022] DELETE /identity/v3/projects/2a230cac62c54799bd5c4d7549390963 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:17.685865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.686163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.686454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.706343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:17.707549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:17.768960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1269/2537] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:17.776560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.776892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.777310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.797304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:17.798490 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:17.806176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1269/2538] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:17.810316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:17.819996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f7ebf-4768-454a-b07f-40147d4aecae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:17.824865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:17.825469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.825741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.826007 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.826674 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:17.827020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:17.828777 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:17.840423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1270/2539] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:54:17.845355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:17.854609 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a984eb4-1f51-4996-92b8-e40ebde59b74 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:17.860062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:17.860674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.860951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.861222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.861850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:17.863738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:17.891631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1270/2540] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:17 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:17.895955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:17.906216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5847fb-2258-4000-968e-f8d50b9329b2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:17.949569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:17.962838 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:17.963451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:17.963728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:17.964028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:17.964668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:17.966644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:17.988365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1271/2541] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:17 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:17.992779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.003429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ca591-1ad6-479c-8e65-1348cd0aa593 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.008070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.008666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.008983 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.009205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.009835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.011412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.012672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:18.023537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1271/2542] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:18.027480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.037275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9547ea-a504-4b35-b5ff-40bfabc8cd4a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.042028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7dc56e-7a6c-4104-bdd2-ced2bf84333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.042628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dc56e-7a6c-4104-bdd2-ced2bf84333c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.042903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dc56e-7a6c-4104-bdd2-ced2bf84333c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.043172 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7dc56e-7a6c-4104-bdd2-ced2bf84333c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.043846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7dc56e-7a6c-4104-bdd2-ced2bf84333c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.045459 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7dc56e-7a6c-4104-bdd2-ced2bf84333c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.057523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1272/2543] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:18.061535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.072553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dca88-7529-437f-bb77-971b12666c90 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.077225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.078013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.078292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.078563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] PATH_INFO: `/v3/projects/7b5f54e42dae479282e5ba7854c3ea1b/users/0c6b590c555b4690a5d6cc3c8f89103a/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.093775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5f54e42dae479282e5ba7854c3ea1b, user_id=0c6b590c555b4690a5d6cc3c8f89103a, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.096685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.111687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1272/2544] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:18 2022] PUT /identity/v3/projects/7b5f54e42dae479282e5ba7854c3ea1b/users/0c6b590c555b4690a5d6cc3c8f89103a/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:18.123432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.123632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.123888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.163602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] MFA Rules not processed for user `0c6b590c555b4690a5d6cc3c8f89103a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:18.164825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:18.245236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1273/2545] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:18.250833 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.321685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3ba7e-5d22-4991-9d1e-8353bafc9e51 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.326989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.327924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.328208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.328483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.329151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.331098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.364512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1273/2546] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:18 2022] POST /identity/v3/projects => generated 389 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:18.369225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.380829 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950511c-691c-4025-812d-7dd1c0eb2ee3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.437914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.438527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.438806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.439076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.439697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.441655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.463032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1274/2547] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:18 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:18.467117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.479124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f3d5-9039-4de0-89f5-c0e78393d95b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.484842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.485635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.485915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.486226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.487057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.488872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.504632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1274/2548] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:18.505913 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.516593 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fe236-2be2-4431-b138-914bbf620a41 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.521726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.522387 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.522665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.522940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] PATH_INFO: `/v3/projects/70485814923e4fa7836dc83c273a4f8c/users/b4c86c9bedf6478fb71ed0689a306781/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.539182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70485814923e4fa7836dc83c273a4f8c, user_id=b4c86c9bedf6478fb71ed0689a306781, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.541950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.556678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1275/2549] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:18 2022] PUT /identity/v3/projects/70485814923e4fa7836dc83c273a4f8c/users/b4c86c9bedf6478fb71ed0689a306781/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:18.560560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.575131 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694a701-8c39-4880-b011-bbf729b28ec0 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.636350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4736baf-7cc3-469e-9bb1-734895c79d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.636871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4736baf-7cc3-469e-9bb1-734895c79d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.637143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4736baf-7cc3-469e-9bb1-734895c79d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.637408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4736baf-7cc3-469e-9bb1-734895c79d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.638041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4736baf-7cc3-469e-9bb1-734895c79d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.639631 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4736baf-7cc3-469e-9bb1-734895c79d72 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.649699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1275/2550] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:18.653966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.665507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659b2b4-f11e-4aac-a3da-cf24060236e4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.670272 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.670929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.671857 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.672138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] PATH_INFO: `/v3/projects/70485814923e4fa7836dc83c273a4f8c/users/b4c86c9bedf6478fb71ed0689a306781/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.674060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=70485814923e4fa7836dc83c273a4f8c, user_id=b4c86c9bedf6478fb71ed0689a306781, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.676775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.688093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1276/2551] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:18 2022] PUT /identity/v3/projects/70485814923e4fa7836dc83c273a4f8c/users/b4c86c9bedf6478fb71ed0689a306781/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:18.698310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.698587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.698855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.727199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] MFA Rules not processed for user `b4c86c9bedf6478fb71ed0689a306781`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:18.728446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:18.759405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:18.803173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1276/2552] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:18 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:18.815153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:18.816945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:18.827782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:18.879612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28eade8-b974-482e-ad28-79e7eb36d64f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:18.883364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f450e-fbde-48ce-8d45-17c8e01d6c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:18.884008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f450e-fbde-48ce-8d45-17c8e01d6c44 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:18.884286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f450e-fbde-48ce-8d45-17c8e01d6c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:18.884564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f450e-fbde-48ce-8d45-17c8e01d6c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:18.894200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f450e-fbde-48ce-8d45-17c8e01d6c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:18.895565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f450e-fbde-48ce-8d45-17c8e01d6c44 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:18.905223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1277/2553] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:18 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:19.063226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:19.147011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4af96f-c067-491e-be64-72a3595274d5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:19.153365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:19.154012 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:19.154277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:19.154543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] PATH_INFO: `/v3/users/0c6b590c555b4690a5d6cc3c8f89103a` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:19.156222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c6b590c555b4690a5d6cc3c8f89103a)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:19.158181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:19.205841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c6b590c555b4690a5d6cc3c8f89103a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:19.213059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c6b590c555b4690a5d6cc3c8f89103a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:19.220965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c6b590c555b4690a5d6cc3c8f89103a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:19.221724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c6b590c555b4690a5d6cc3c8f89103a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:19.237838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c6b590c555b4690a5d6cc3c8f89103a'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:19.249154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1277/2554] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:19 2022] DELETE /identity/v3/users/0c6b590c555b4690a5d6cc3c8f89103a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:19.250804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:19.251259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:19.251385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:19.281630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:19.283335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:19.362236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1278/2555] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:54:19.375814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:19.377483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:19.434706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:19.441193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02beaf5-11e6-4751-a09e-8b28a55cf551 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:19.446072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:19.446777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:19.447098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:19.447368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:19.456980 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:19.458648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:19.470818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1278/2556] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:20.062595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:20.064257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:20.076626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9fb92-3ee6-4554-a7d6-0592848274d5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:20.083885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:20.084561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.084831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.085103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] PATH_INFO: `/v3/projects/7b5f54e42dae479282e5ba7854c3ea1b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.086223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5f54e42dae479282e5ba7854c3ea1b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:20.088166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:20.136609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5f54e42dae479282e5ba7854c3ea1b', 'request_id': 'req-eb969486-5de2-4228-81ae-81fd54f6a8d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.150101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5f54e42dae479282e5ba7854c3ea1b', 'request_id': 'req-eb969486-5de2-4228-81ae-81fd54f6a8d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.157574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5f54e42dae479282e5ba7854c3ea1b', 'request_id': 'req-eb969486-5de2-4228-81ae-81fd54f6a8d1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.169105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] The token cache is being invalidate because project 7b5f54e42dae479282e5ba7854c3ea1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:20.169398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.170078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.171664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1279/2557] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:20 2022] DELETE /identity/v3/projects/7b5f54e42dae479282e5ba7854c3ea1b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:20.175867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:20.185671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ce595-f0b1-4ffe-b5df-aa31e542518f service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:20.243514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:20.244188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.244467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.244742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] PATH_INFO: `/v3/users/b4c86c9bedf6478fb71ed0689a306781` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.245869 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4c86c9bedf6478fb71ed0689a306781)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:20.247858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:20.296081 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4c86c9bedf6478fb71ed0689a306781'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.305183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4c86c9bedf6478fb71ed0689a306781'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.313519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4c86c9bedf6478fb71ed0689a306781'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.314201 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4c86c9bedf6478fb71ed0689a306781'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.327078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-87b2dbaa-e4cc-4c31-a5ea-1ce093706a71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4c86c9bedf6478fb71ed0689a306781'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.333141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1279/2558] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:20 2022] DELETE /identity/v3/users/b4c86c9bedf6478fb71ed0689a306781 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:20.657129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:20.658575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:20.674705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb969486-5de2-4228-81ae-81fd54f6a8d1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:20.707558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:20.734292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:20.734977 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.735257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.735528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] PATH_INFO: `/v3/projects/70485814923e4fa7836dc83c273a4f8c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.736678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70485814923e4fa7836dc83c273a4f8c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:20.738761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:20.784162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70485814923e4fa7836dc83c273a4f8c', 'request_id': 'req-154725b8-0546-4733-adb5-d57bbfa5793f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.791552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70485814923e4fa7836dc83c273a4f8c', 'request_id': 'req-154725b8-0546-4733-adb5-d57bbfa5793f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.798474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70485814923e4fa7836dc83c273a4f8c', 'request_id': 'req-154725b8-0546-4733-adb5-d57bbfa5793f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.809202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] The token cache is being invalidate because project 70485814923e4fa7836dc83c273a4f8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:20.809484 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.810107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-154725b8-0546-4733-adb5-d57bbfa5793f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:20.811560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1280/2559] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:20 2022] DELETE /identity/v3/projects/70485814923e4fa7836dc83c273a4f8c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:20.816634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.816911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.817181 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.837774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:20.838958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:20.897416 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1280/2560] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:20.905038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.905385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.906225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.926590 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:20.927866 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:20.936015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1281/2561] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:20.940343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:20.949799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0a7d4-3dc2-4634-a51b-e4883e5e69bf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:20.954443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:20.955077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.955357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.955633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.956301 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:20.956608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:20.958183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:20.960578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:20.975334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1281/2562] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:54:20.976612 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:20.986754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37522831-7f2d-430b-9a6d-d1cafe64aa2e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:20.991400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:20.992293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:20.992560 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:20.992824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:20.993441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:20.995341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.022636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1282/2563] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:20 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:21.026925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.037618 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82065d58-27c9-4660-8727-636d4bb3f781 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.095253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.095892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.096208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.096444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.097083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.099021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.120393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1282/2564] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:21 2022] POST /identity/v3/users => generated 304 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:21.124313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.134631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58821d74-44d4-40cf-8b9f-661c011ddcfc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.139322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.140474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.140749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.141020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.141657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.143258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.153357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1283/2565] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:21.157967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.168142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8e634-6751-4d7f-a71e-8258b25bd4d8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.172693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487163-65f4-49fc-bfeb-5467cbd13dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.173281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487163-65f4-49fc-bfeb-5467cbd13dfb admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.173556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487163-65f4-49fc-bfeb-5467cbd13dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.173823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487163-65f4-49fc-bfeb-5467cbd13dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.174446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d487163-65f4-49fc-bfeb-5467cbd13dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.176051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d487163-65f4-49fc-bfeb-5467cbd13dfb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.186402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1283/2566] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:21.190627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.200345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83091af-dc0b-4371-a3fe-a4745f4e3ca9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.205385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.206021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.206296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.206566 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] PATH_INFO: `/v3/projects/f7e82fbc96114cde8c78cebe23f5e62d/users/fef9b55b93bb4bafa3ffe9764bbb71c1/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.221334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e82fbc96114cde8c78cebe23f5e62d, user_id=fef9b55b93bb4bafa3ffe9764bbb71c1, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.224036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.236532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1284/2567] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:21 2022] PUT /identity/v3/projects/f7e82fbc96114cde8c78cebe23f5e62d/users/fef9b55b93bb4bafa3ffe9764bbb71c1/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:21.247808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.248087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.248355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.276195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] MFA Rules not processed for user `fef9b55b93bb4bafa3ffe9764bbb71c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:21.277421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:21.340434 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1284/2568] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:21 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:21.345615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.407040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c4603-7f6d-4a75-9da1-cf50c7199e20 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.411713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.412335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.412608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.412889 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.413522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.415434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.420636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:21.444469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1285/2569] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:21 2022] POST /identity/v3/projects => generated 409 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:21.448808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.459884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee54a-379f-4b3a-a233-f33863755feb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.514365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.515567 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.515861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.516123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.516761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.518623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.528484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:21.540378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1285/2570] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:21 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:21.544583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.554662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9b92f-6769-447d-9623-a7aa2e9e4d7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.559158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.559767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.560059 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.560331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.560965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.563105 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.572778 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1286/2571] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:21.577046 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.587832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ac726-d93f-4ec8-a976-ad51ca85f74d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.592471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.593052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.593330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.593602 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] PATH_INFO: `/v3/projects/88adbd14a7ee4e759ab8c27677eec01c/users/1cde1690af804774b43a2fe5f3429d75/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.608728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=88adbd14a7ee4e759ab8c27677eec01c, user_id=1cde1690af804774b43a2fe5f3429d75, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.611377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.627823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1286/2572] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:21 2022] PUT /identity/v3/projects/88adbd14a7ee4e759ab8c27677eec01c/users/1cde1690af804774b43a2fe5f3429d75/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:21.628972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.639482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a7fd7-9bb0-42ff-853f-f5f116c8103a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.697540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525c62ff-6c74-4afd-8a39-4934245411f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.698240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c62ff-6c74-4afd-8a39-4934245411f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.698415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c62ff-6c74-4afd-8a39-4934245411f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.698687 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525c62ff-6c74-4afd-8a39-4934245411f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.699328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c62ff-6c74-4afd-8a39-4934245411f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.700962 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525c62ff-6c74-4afd-8a39-4934245411f3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.711996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1287/2573] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:21.716336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.727072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5839b22-00eb-4593-b7fc-c3bec15b4d1d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.731933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.732730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.732850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.733123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] PATH_INFO: `/v3/projects/88adbd14a7ee4e759ab8c27677eec01c/users/1cde1690af804774b43a2fe5f3429d75/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.735010 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] RBAC: Authorizing `identity:create_grant(project_id=88adbd14a7ee4e759ab8c27677eec01c, user_id=1cde1690af804774b43a2fe5f3429d75, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.737695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.749261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1287/2574] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:21 2022] PUT /identity/v3/projects/88adbd14a7ee4e759ab8c27677eec01c/users/1cde1690af804774b43a2fe5f3429d75/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:21.760250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.760538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.760813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.789274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] MFA Rules not processed for user `1cde1690af804774b43a2fe5f3429d75`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:21.790474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:21.858433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1288/2575] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:21 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:21.870085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:21.871845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:21.929705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4bc3b-d6a1-46ff-9f76-09aef939107e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:21.933310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:21.933912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:21.934184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:21.934451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:21.943423 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:21.944871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:21.954575 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1288/2576] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:21 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:22.103271 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:22.113238 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fa20e-9664-4f63-9ec8-dd8de2dd62e3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:22.116967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0952-5655-46f9-8d51-5d09423aca52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:22.117556 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0952-5655-46f9-8d51-5d09423aca52 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:22.117831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0952-5655-46f9-8d51-5d09423aca52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:22.118091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0952-5655-46f9-8d51-5d09423aca52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:22.173660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:22.176087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0952-5655-46f9-8d51-5d09423aca52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:22.178613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0952-5655-46f9-8d51-5d09423aca52 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:22.188397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1289/2577] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:22 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:27.477964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:27.541799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ee143-8221-4683-b570-b9ab0d75d258 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:27.546404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:27.547555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:27.547872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:27.548147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] PATH_INFO: `/v3/users/fef9b55b93bb4bafa3ffe9764bbb71c1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:27.549287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef9b55b93bb4bafa3ffe9764bbb71c1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:27.551188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:27.599201 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef9b55b93bb4bafa3ffe9764bbb71c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:27.607056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef9b55b93bb4bafa3ffe9764bbb71c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:27.613693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:27.615795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef9b55b93bb4bafa3ffe9764bbb71c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:27.617260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef9b55b93bb4bafa3ffe9764bbb71c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:27.632398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef9b55b93bb4bafa3ffe9764bbb71c1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:27.638310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1289/2578] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:27 2022] DELETE /identity/v3/users/fef9b55b93bb4bafa3ffe9764bbb71c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:27.642965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:27.643495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:27.643629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:27.663078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:27.664310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:27.724377 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1290/2579] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:54:27.735160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:27.736689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:27.794784 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dc05a-51ef-4a15-ac76-89ca14f66e4e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:27.798352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:27.798956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:27.799229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:27.799497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:27.808342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:27.809754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:27.819802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1290/2580] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:28.088320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:28.089759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:28.101704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef416cf3-926c-438a-8022-a26e2a181fd7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:28.110186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:28.110840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.111116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.111389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] PATH_INFO: `/v3/projects/f7e82fbc96114cde8c78cebe23f5e62d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.112581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e82fbc96114cde8c78cebe23f5e62d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:28.114524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:28.161743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e82fbc96114cde8c78cebe23f5e62d', 'request_id': 'req-2686f198-6089-4efa-b25e-b8102f04067c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.169872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e82fbc96114cde8c78cebe23f5e62d', 'request_id': 'req-2686f198-6089-4efa-b25e-b8102f04067c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.176253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e82fbc96114cde8c78cebe23f5e62d', 'request_id': 'req-2686f198-6089-4efa-b25e-b8102f04067c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.187158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] The token cache is being invalidate because project f7e82fbc96114cde8c78cebe23f5e62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:28.187444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.188093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.189562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1291/2581] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:28 2022] DELETE /identity/v3/projects/f7e82fbc96114cde8c78cebe23f5e62d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:28.193571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:28.203084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990a83-665c-46da-a246-5e5f088f30a3 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:28.261300 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:28.261929 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.262208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.262478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] PATH_INFO: `/v3/users/1cde1690af804774b43a2fe5f3429d75` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.263590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cde1690af804774b43a2fe5f3429d75)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:28.265674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:28.310256 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cde1690af804774b43a2fe5f3429d75'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.317731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cde1690af804774b43a2fe5f3429d75'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.324628 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cde1690af804774b43a2fe5f3429d75'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.325260 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cde1690af804774b43a2fe5f3429d75'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.337641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-1e7b3917-1445-4249-a382-4df9380eb795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cde1690af804774b43a2fe5f3429d75'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.339728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:28.344768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1291/2582] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:28 2022] DELETE /identity/v3/users/1cde1690af804774b43a2fe5f3429d75 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:28.617311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:28.618945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:28.633291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686f198-6089-4efa-b25e-b8102f04067c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:28.694268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:28.694912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.695167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.695435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] PATH_INFO: `/v3/projects/88adbd14a7ee4e759ab8c27677eec01c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.696608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88adbd14a7ee4e759ab8c27677eec01c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:28.698548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:28.724584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:28.750533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88adbd14a7ee4e759ab8c27677eec01c', 'request_id': 'req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.758634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88adbd14a7ee4e759ab8c27677eec01c', 'request_id': 'req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.764976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88adbd14a7ee4e759ab8c27677eec01c', 'request_id': 'req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.778141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] The token cache is being invalidate because project 88adbd14a7ee4e759ab8c27677eec01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:28.778535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.782199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b5f46f7-6d1a-42d7-8a7d-8de06000ea53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:28.783823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1292/2583] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:28 2022] DELETE /identity/v3/projects/88adbd14a7ee4e759ab8c27677eec01c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:28.788837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.789296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.789489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.815178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:28.816534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:28.891892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1292/2584] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:28.895160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.895443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.895718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.915966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:28.917171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:28.925569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1293/2585] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:28.930217 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:28.940754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760604eb-2f21-42fa-bc22-9841e90aa6a3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:28.945522 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:28.946067 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.946352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.946621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.947263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:28.947565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:28.949176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:28.961412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1293/2586] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:54:28.965841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:28.975018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ed1ec-c28c-459a-b8e8-c3c9ed33a416 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:28.979598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:28.980227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:28.980507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:28.980776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:28.981395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:28.983297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.010403 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1294/2587] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:28 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:29.014527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.025078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db2e3a-f29f-4499-b16a-2d7408b9ffe9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.082177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.082790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.083061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.083334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.083990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.085912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.106953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1294/2588] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:29.111382 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.121966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e09fbf-9933-4aa0-97fa-ed50967c2d7c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.128749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.129420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.129960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.130275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.130957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.132599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.146529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1295/2589] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:54:29.147879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.158195 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033b885-e452-4960-9a00-79a119877682 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.162858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294d7a4-ad91-4643-9694-969d68f7b961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.163460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294d7a4-ad91-4643-9694-969d68f7b961 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.163871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294d7a4-ad91-4643-9694-969d68f7b961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.164141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294d7a4-ad91-4643-9694-969d68f7b961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.164921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294d7a4-ad91-4643-9694-969d68f7b961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.166518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294d7a4-ad91-4643-9694-969d68f7b961 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.177587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1295/2590] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:29.181544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.192090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6de519-576a-4bcf-8c97-b4d961fffbd8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.196750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.197332 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.197614 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.197879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] PATH_INFO: `/v3/projects/2c3a78f00b224cb7b31f407d54abce89/users/27a6fa23cd114ca4ba7129ab3914a981/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.214525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3a78f00b224cb7b31f407d54abce89, user_id=27a6fa23cd114ca4ba7129ab3914a981, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.217226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.231264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1296/2591] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:29 2022] PUT /identity/v3/projects/2c3a78f00b224cb7b31f407d54abce89/users/27a6fa23cd114ca4ba7129ab3914a981/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:29.241770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.242312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.242752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.273964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] MFA Rules not processed for user `27a6fa23cd114ca4ba7129ab3914a981`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:29.275202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:29.333421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:29.343808 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1296/2592] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:29.348279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.411561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7580af6-c48f-4648-8f91-503f69d2d945 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.416320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.416921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.417197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.417465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.418203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.420358 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.447529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1297/2593] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:29.451629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.462223 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e9a5c-3d1a-44b5-b439-67d257507e5c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.518577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.519190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.519468 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.519738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.520404 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.522308 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.542393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1297/2594] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:29.546998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.557112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb975b-7c5e-4f63-bd1f-eb952404a244 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.561843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.563070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.563344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.563609 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.564339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.566581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.576394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1298/2595] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:29.580396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.590333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01525112-5c56-489f-98a8-7e255a97d70b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.594971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2500ba0-41fb-4664-913f-5c03423ea157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.595562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500ba0-41fb-4664-913f-5c03423ea157 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.595947 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500ba0-41fb-4664-913f-5c03423ea157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.596158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2500ba0-41fb-4664-913f-5c03423ea157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.596795 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500ba0-41fb-4664-913f-5c03423ea157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.598377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2500ba0-41fb-4664-913f-5c03423ea157 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.608173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1298/2596] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:29.612705 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.622875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956af4b8-5682-4762-bc69-e0859308185c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.628222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.628904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.629215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.629565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] PATH_INFO: `/v3/projects/560d63059b214d6c9adeb6fd74dd7acd/users/ba76d4b6674b40f2842af2a009bc24b1/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.644221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=560d63059b214d6c9adeb6fd74dd7acd, user_id=ba76d4b6674b40f2842af2a009bc24b1, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.646859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.649944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:29.662157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1299/2597] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:29 2022] PUT /identity/v3/projects/560d63059b214d6c9adeb6fd74dd7acd/users/ba76d4b6674b40f2842af2a009bc24b1/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:29.673109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.673395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.673677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.700889 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] MFA Rules not processed for user `ba76d4b6674b40f2842af2a009bc24b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:29.701998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:29.769612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1299/2598] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:29.774290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.837630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1da06-28a6-4331-a07d-ecd9709193e8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.842252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.842845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.843115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.843911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.844533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.846425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.874702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1300/2599] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:29.878848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.889693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e282ea-a8ea-49bb-8342-b38969d99356 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.950061 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.950653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.950979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.951194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.951875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:29.953788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:29.964059 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:29.977692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1300/2600] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:29 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:29.981640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:29.992535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf1053-73de-4fb7-bd1b-f88f83378df7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:29.997498 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:29.998091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:29.998360 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:29.998623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:29.999261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:30.000925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:30.011762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1301/2601] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:30.017209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:30.027327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eadd6d-7c3a-458f-acf1-ef84890ad4da admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:30.033011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:30.033689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:30.033964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:30.034237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] PATH_INFO: `/v3/projects/babf74e9ab8c4f108b3ce672b15a98c0/users/bbd497be861b4e25b4df6ad068b1e734/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:30.049710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=babf74e9ab8c4f108b3ce672b15a98c0, user_id=bbd497be861b4e25b4df6ad068b1e734, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:30.052402 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:30.067163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1301/2602] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:30 2022] PUT /identity/v3/projects/babf74e9ab8c4f108b3ce672b15a98c0/users/bbd497be861b4e25b4df6ad068b1e734/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:30.070888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:30.081538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103f1f1-8b7c-4de8-aff4-08dbbfa2f0ee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:30.142182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737074b-12f8-45a2-aed0-587b23a1ce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:30.142896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737074b-12f8-45a2-aed0-587b23a1ce01 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:30.143173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737074b-12f8-45a2-aed0-587b23a1ce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:30.143445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737074b-12f8-45a2-aed0-587b23a1ce01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:30.144128 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737074b-12f8-45a2-aed0-587b23a1ce01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:30.145756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737074b-12f8-45a2-aed0-587b23a1ce01 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:30.159807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1302/2603] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:30.161072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:30.171555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff9b6-d5fd-423a-9d0b-e2a80f073d8b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:30.176295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:30.177022 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:30.177369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:30.177708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] PATH_INFO: `/v3/projects/babf74e9ab8c4f108b3ce672b15a98c0/users/bbd497be861b4e25b4df6ad068b1e734/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:30.179684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=babf74e9ab8c4f108b3ce672b15a98c0, user_id=bbd497be861b4e25b4df6ad068b1e734, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:30.182385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:30.192882 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1302/2604] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:30 2022] PUT /identity/v3/projects/babf74e9ab8c4f108b3ce672b15a98c0/users/bbd497be861b4e25b4df6ad068b1e734/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:30.261775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:30.262226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:30.262774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:30.291532 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] MFA Rules not processed for user `bbd497be861b4e25b4df6ad068b1e734`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:30.292803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:30.346489 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:30.365652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1303/2605] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:30.377167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:30.378918 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:30.438412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853bd6c-3432-4a2a-8681-5bd9d54b8034 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:30.442326 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:30.442937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:30.443257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:30.443470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:30.453835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:30.455291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:30.464844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1303/2606] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:30.507094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:30.508492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:30.520754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a343-2095-43f3-a180-4afc9087ed55 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:30.524337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f83fb7-d156-4cf0-8a1f-1aaf8d89f757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:30.524931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83fb7-d156-4cf0-8a1f-1aaf8d89f757 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:30.525227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83fb7-d156-4cf0-8a1f-1aaf8d89f757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:30.525491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f83fb7-d156-4cf0-8a1f-1aaf8d89f757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:30.583370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83fb7-d156-4cf0-8a1f-1aaf8d89f757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:30.584871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f83fb7-d156-4cf0-8a1f-1aaf8d89f757 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:30.593864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1304/2607] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:33.526932 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:33.589600 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1b01e-f583-473e-8ab6-1293c6ec0b03 service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:33.595141 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:33.595790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:33.596055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:33.596323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] PATH_INFO: `/v3/users/27a6fa23cd114ca4ba7129ab3914a981` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:33.597420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] RBAC: Authorizing `identity:delete_user(user_id=27a6fa23cd114ca4ba7129ab3914a981)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:33.599292 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:33.645691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27a6fa23cd114ca4ba7129ab3914a981'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:33.652964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27a6fa23cd114ca4ba7129ab3914a981'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:33.658805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:33.660839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27a6fa23cd114ca4ba7129ab3914a981'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:33.661724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27a6fa23cd114ca4ba7129ab3914a981'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:33.675928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27a6fa23cd114ca4ba7129ab3914a981'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:33.681864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1304/2608] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:33 2022] DELETE /identity/v3/users/27a6fa23cd114ca4ba7129ab3914a981 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:33.686253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ed113-817c-4b2f-9027-069590562e9b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:33.686762 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ed113-817c-4b2f-9027-069590562e9b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:33.686874 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ed113-817c-4b2f-9027-069590562e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:33.705042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-840ed113-817c-4b2f-9027-069590562e9b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:33.706222 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-840ed113-817c-4b2f-9027-069590562e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:33.766154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1305/2609] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:33.776129 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:33.777515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:33.835494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5596cc-d661-4543-a658-31416835f40c admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:33.839370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:33.840088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:33.840407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:33.840789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:33.852499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:33.853927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:33.863429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1305/2610] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:34.135897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ed113-817c-4b2f-9027-069590562e9b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:34.137526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-840ed113-817c-4b2f-9027-069590562e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:34.150649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ed113-817c-4b2f-9027-069590562e9b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:34.156386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:34.157048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:34.157326 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:34.157597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] PATH_INFO: `/v3/projects/2c3a78f00b224cb7b31f407d54abce89` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:34.158734 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3a78f00b224cb7b31f407d54abce89)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:34.160681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:34.211853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3a78f00b224cb7b31f407d54abce89', 'request_id': 'req-693c4b58-7f5d-4838-b68d-f7c225fb090d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.220022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3a78f00b224cb7b31f407d54abce89', 'request_id': 'req-693c4b58-7f5d-4838-b68d-f7c225fb090d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.226190 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3a78f00b224cb7b31f407d54abce89', 'request_id': 'req-693c4b58-7f5d-4838-b68d-f7c225fb090d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.237220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] The token cache is being invalidate because project 2c3a78f00b224cb7b31f407d54abce89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:34.237501 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.238155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.239616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1306/2611] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:34 2022] DELETE /identity/v3/projects/2c3a78f00b224cb7b31f407d54abce89 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:34.243956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:34.254408 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a23b18-12c6-4cfd-bd3d-da23e95ea541 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:34.308972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:34.309587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:34.309857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:34.310124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] PATH_INFO: `/v3/users/ba76d4b6674b40f2842af2a009bc24b1` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:34.311212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba76d4b6674b40f2842af2a009bc24b1)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:34.313708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:34.358883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba76d4b6674b40f2842af2a009bc24b1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.366388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba76d4b6674b40f2842af2a009bc24b1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.373726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba76d4b6674b40f2842af2a009bc24b1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.374979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba76d4b6674b40f2842af2a009bc24b1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.386786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba76d4b6674b40f2842af2a009bc24b1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.388874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:34.394044 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1306/2612] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:34 2022] DELETE /identity/v3/users/ba76d4b6674b40f2842af2a009bc24b1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:34.668204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:34.670304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:34.692586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c4b58-7f5d-4838-b68d-f7c225fb090d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:34.710136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:34.759211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:34.759791 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:34.760071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:34.760342 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] PATH_INFO: `/v3/projects/560d63059b214d6c9adeb6fd74dd7acd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:34.761443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=560d63059b214d6c9adeb6fd74dd7acd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:34.763340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:34.809976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '560d63059b214d6c9adeb6fd74dd7acd', 'request_id': 'req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.818076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '560d63059b214d6c9adeb6fd74dd7acd', 'request_id': 'req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.823764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '560d63059b214d6c9adeb6fd74dd7acd', 'request_id': 'req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.834911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] The token cache is being invalidate because project 560d63059b214d6c9adeb6fd74dd7acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:34.835198 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.835854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.837329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1307/2613] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:34 2022] DELETE /identity/v3/projects/560d63059b214d6c9adeb6fd74dd7acd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:34.841324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:34.842754 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:34.855320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534935d0-b291-40ae-9560-884a61ee5234 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:34.911481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:34.912145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:34.912412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:34.912683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] PATH_INFO: `/v3/users/bbd497be861b4e25b4df6ad068b1e734` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:34.913959 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbd497be861b4e25b4df6ad068b1e734)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:34.915891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:34.963823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbd497be861b4e25b4df6ad068b1e734'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.973505 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbd497be861b4e25b4df6ad068b1e734'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.981847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbd497be861b4e25b4df6ad068b1e734'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.982655 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbd497be861b4e25b4df6ad068b1e734'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:34.996127 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-0e5c8d24-2df7-4f22-826f-a5aacdb0ea27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbd497be861b4e25b4df6ad068b1e734'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:35.005306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1307/2614] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:34 2022] DELETE /identity/v3/users/bbd497be861b4e25b4df6ad068b1e734 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:35.279820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.281870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:35.294523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3e3c2-e405-46a7-bf3f-e9b3323a9d80 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.351052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.351671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.351973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.352241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] PATH_INFO: `/v3/projects/babf74e9ab8c4f108b3ce672b15a98c0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.353338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=babf74e9ab8c4f108b3ce672b15a98c0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.355220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.398704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:35.401733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'babf74e9ab8c4f108b3ce672b15a98c0', 'request_id': 'req-0ab09789-e80c-4fc1-82eb-4535ad776fd8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:35.410779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'babf74e9ab8c4f108b3ce672b15a98c0', 'request_id': 'req-0ab09789-e80c-4fc1-82eb-4535ad776fd8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:35.416367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'babf74e9ab8c4f108b3ce672b15a98c0', 'request_id': 'req-0ab09789-e80c-4fc1-82eb-4535ad776fd8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:35.427645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] The token cache is being invalidate because project babf74e9ab8c4f108b3ce672b15a98c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:35.427961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:35.428622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0ab09789-e80c-4fc1-82eb-4535ad776fd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:35.430096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1308/2615] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:35 2022] DELETE /identity/v3/projects/babf74e9ab8c4f108b3ce672b15a98c0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:35.436340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.436605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.437513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.457400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:35.458594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:35.520295 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1308/2616] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:35.528640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.528861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.529136 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.548877 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:35.550077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:35.557993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1309/2617] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:35.562766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.572398 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b45ef-ff66-4815-9515-1757ba71278d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.576914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.577528 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.577831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.578101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.578740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:35.579040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.580654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.592149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1309/2618] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:54:35.596472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.605504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f2087-a399-43ca-8485-8e2a3f000e0c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.610867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.611465 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.611768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.612039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.612662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.614563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.642429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1310/2619] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:35 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:35.646769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.657124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69114de9-4217-430a-9a2b-3f256a76f005 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.660671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:35.713827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.714428 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.714703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.714967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.715595 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.717516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.738793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1310/2620] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:35 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:35.748426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.758964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8864c2-f771-4f04-ae52-be5e8e95e290 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.763604 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.764228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.764585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.764860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.765505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.767101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.776969 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1311/2621] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:35.782107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.792859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d96edf-7341-43d0-8f1d-e660d91f5153 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.798327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b359e-bb8c-4100-b29d-5706d3c4b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.798931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b359e-bb8c-4100-b29d-5706d3c4b9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.799207 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b359e-bb8c-4100-b29d-5706d3c4b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.799477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b359e-bb8c-4100-b29d-5706d3c4b9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.800362 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b359e-bb8c-4100-b29d-5706d3c4b9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.801942 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b359e-bb8c-4100-b29d-5706d3c4b9dc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.813155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1311/2622] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:35.817330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.827245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2f160-00da-4c32-80f5-7fdee6915213 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:35.833019 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:35.833665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.834093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.834252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] PATH_INFO: `/v3/projects/316fcd073a59490986e5facf11dc65ae/users/42161bfa7710420e9962b2d7e1e17967/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.848859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=316fcd073a59490986e5facf11dc65ae, user_id=42161bfa7710420e9962b2d7e1e17967, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:35.851493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:35.864296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1312/2623] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:35 2022] PUT /identity/v3/projects/316fcd073a59490986e5facf11dc65ae/users/42161bfa7710420e9962b2d7e1e17967/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:35.877523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:35.878031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:35.878144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:35.906407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] MFA Rules not processed for user `42161bfa7710420e9962b2d7e1e17967`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:35.907614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:35.972283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1312/2624] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:35.983821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:35.985536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:36.042297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56390d91-bb4b-4ec9-b4c8-e7abd3d2fe3b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:36.045932 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddc741-1da9-4cee-b396-bdae4b975766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:36.047037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddc741-1da9-4cee-b396-bdae4b975766 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:36.047313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddc741-1da9-4cee-b396-bdae4b975766 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:36.047585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddc741-1da9-4cee-b396-bdae4b975766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:36.056565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddc741-1da9-4cee-b396-bdae4b975766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:36.058006 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddc741-1da9-4cee-b396-bdae4b975766 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:36.068216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1313/2625] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:45.969414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:46.063426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:49.592276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:49.660284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472a14d-dc93-40d7-b1cf-9b624c650c8c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:49.665588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:49.666327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:49.666641 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:49.666976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] PATH_INFO: `/v3/users/42161bfa7710420e9962b2d7e1e17967` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:49.668200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] RBAC: Authorizing `identity:delete_user(user_id=42161bfa7710420e9962b2d7e1e17967)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:49.670164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:49.716592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42161bfa7710420e9962b2d7e1e17967'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:49.725752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42161bfa7710420e9962b2d7e1e17967'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:49.734135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42161bfa7710420e9962b2d7e1e17967'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:49.734817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42161bfa7710420e9962b2d7e1e17967'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:49.747187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42161bfa7710420e9962b2d7e1e17967'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:49.753397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1313/2626] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:49 2022] DELETE /identity/v3/users/42161bfa7710420e9962b2d7e1e17967 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:49.758182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:49.758960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:49.758960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:49.779856 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:49.781083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:49.845800 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1314/2627] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:49.855993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:49.857420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:49.913632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99161b9-41b6-49e2-8a95-12fb38c3143e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:49.917120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f433f-57a0-4884-bae3-ed34f065ae63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:49.917714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f433f-57a0-4884-bae3-ed34f065ae63 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:49.917987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f433f-57a0-4884-bae3-ed34f065ae63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:49.918252 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f433f-57a0-4884-bae3-ed34f065ae63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:49.927844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f433f-57a0-4884-bae3-ed34f065ae63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:49.929262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f433f-57a0-4884-bae3-ed34f065ae63 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:49.939345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1314/2628] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:50.202215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.204399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:50.216431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be34c3-9b59-4709-b264-de411f55dae9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.221138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.221764 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.222034 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.222300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] PATH_INFO: `/v3/projects/316fcd073a59490986e5facf11dc65ae` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.223472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316fcd073a59490986e5facf11dc65ae)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.225322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.273565 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316fcd073a59490986e5facf11dc65ae', 'request_id': 'req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:50.282046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316fcd073a59490986e5facf11dc65ae', 'request_id': 'req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:50.287707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316fcd073a59490986e5facf11dc65ae', 'request_id': 'req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:50.299063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] The token cache is being invalidate because project 316fcd073a59490986e5facf11dc65ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:50.299379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:50.300028 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b3fbc39-f4d6-4f79-acb8-9e3610960c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:50.301507 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1315/2629] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:50 2022] DELETE /identity/v3/projects/316fcd073a59490986e5facf11dc65ae => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:50.306450 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.306735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.307025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.327275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:50.328518 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:50.345286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:50.390453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1315/2630] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:54:50.398264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.398543 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.398836 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.418240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:50.419425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:50.427758 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1316/2631] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:50.431943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.442246 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a1b64-6fc0-4b08-ba0d-5517626b7efa None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.446820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.447356 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.447632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.447927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.448558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:54:50.448851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.450442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.462175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1316/2632] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:54:50.466322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.475965 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e681d1b-f225-4a3c-830a-fee090251527 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.480697 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.481294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.481570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.481840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.482508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.484441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.512752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1317/2633] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:54:50 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:54:50.517035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.527248 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467835a3-0a20-4851-b990-86999b283512 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.583640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.584304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.584572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.584843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.585476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.587376 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.607974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1317/2634] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:54:50 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:54:50.612513 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.622753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7924d-ada5-40a7-b110-fb51e70cde1e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.627998 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.628580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.628850 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.629115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.629744 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.631319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.640928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1318/2635] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:50.644772 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.654827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb6a6f-9860-49c3-a280-fd4b46d87890 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.660188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b38f4-a49d-41bb-9435-b857fe94bdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.660783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b38f4-a49d-41bb-9435-b857fe94bdda admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.661060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b38f4-a49d-41bb-9435-b857fe94bdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.661329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b38f4-a49d-41bb-9435-b857fe94bdda admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.661970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b38f4-a49d-41bb-9435-b857fe94bdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.663550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b38f4-a49d-41bb-9435-b857fe94bdda admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.674452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1318/2636] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:54:50.678557 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.688390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749a80df-b340-4489-9c06-8a08d12ecbef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.692991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.693651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.693908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.694176 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] PATH_INFO: `/v3/projects/33141eea891649bd8951f529f7c81522/users/a8de353c3665422fae6973aad569be4f/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.708494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33141eea891649bd8951f529f7c81522, user_id=a8de353c3665422fae6973aad569be4f, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.711067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.724380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1319/2637] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:54:50 2022] PUT /identity/v3/projects/33141eea891649bd8951f529f7c81522/users/a8de353c3665422fae6973aad569be4f/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:54:50.734605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.735531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.735837 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.766483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] MFA Rules not processed for user `a8de353c3665422fae6973aad569be4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:50.767713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:50.831233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1319/2638] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:50.842746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:50.844952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:50.867316 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:50.904550 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a14897-dd8d-46c6-ba0a-34b1fecd3693 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:50.908665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00da958-24b8-4ed3-b0ec-821444256c43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:50.909265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00da958-24b8-4ed3-b0ec-821444256c43 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:50.909535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00da958-24b8-4ed3-b0ec-821444256c43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:50.909801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00da958-24b8-4ed3-b0ec-821444256c43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:50.918241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00da958-24b8-4ed3-b0ec-821444256c43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:50.919679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00da958-24b8-4ed3-b0ec-821444256c43 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:50.930433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1320/2639] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:53.610625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:53.635410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:53.686480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67550c1-2e9f-4181-9db9-66f8995c36f3 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:53.692158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:53.692890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:53.693176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:53.693444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] PATH_INFO: `/v3/users/a8de353c3665422fae6973aad569be4f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:53.694566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8de353c3665422fae6973aad569be4f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:53.696577 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:53.746000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8de353c3665422fae6973aad569be4f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:53.753436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8de353c3665422fae6973aad569be4f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:53.760925 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8de353c3665422fae6973aad569be4f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:53.761611 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8de353c3665422fae6973aad569be4f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:53.773783 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8de353c3665422fae6973aad569be4f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:53.780101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1320/2640] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:53 2022] DELETE /identity/v3/users/a8de353c3665422fae6973aad569be4f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:53.785206 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:53.785691 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:53.785816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:53.807940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:54:53.809644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:53.884279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1321/2641] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:54:53.894001 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:53.895502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:53.962955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40534e-4913-4b21-902f-076dac057bb1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:53.966705 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:53.967313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:53.967589 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:53.967898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:53.978182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:53.979616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:53.990334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1321/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:54:54.275036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:54.276480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:54.295185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a8068-ca53-4718-980a-996b2f7f6f04 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:54.301474 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:54.302207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:54.302546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:54.302905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] PATH_INFO: `/v3/projects/33141eea891649bd8951f529f7c81522` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:54.304096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33141eea891649bd8951f529f7c81522)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:54.306077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:54.358035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33141eea891649bd8951f529f7c81522', 'request_id': 'req-81fc4b85-0bed-421a-82af-ecd36e4f1b89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:54.366303 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33141eea891649bd8951f529f7c81522', 'request_id': 'req-81fc4b85-0bed-421a-82af-ecd36e4f1b89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:54.372942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33141eea891649bd8951f529f7c81522', 'request_id': 'req-81fc4b85-0bed-421a-82af-ecd36e4f1b89'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:54.385715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] The token cache is being invalidate because project 33141eea891649bd8951f529f7c81522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:54:54.386033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:54.386690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:54:54.388204 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1322/2643] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:54:54 2022] DELETE /identity/v3/projects/33141eea891649bd8951f529f7c81522 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:54:56.737803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:56.739199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:56.751043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320e2ba-86fa-47b4-a4e0-9c9c45cf5404 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:56.788632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:54:56.800729 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:56.801330 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:56.801606 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:56.801878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:56.803564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:56.866057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:56.867482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:54:56.877908 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1322/2644] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:54:59.936339 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:54:59.938390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:54:59.958633 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc4b85-0bed-421a-82af-ecd36e4f1b89 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:54:59.963620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:54:59.964319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:54:59.964582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:54:59.964855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] PATH_INFO: `/v3/users/cdb5ce192c764fe4946542f5960ccd39` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:54:59.966001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb5ce192c764fe4946542f5960ccd39)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:54:59.968094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.009595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:00.015898 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb5ce192c764fe4946542f5960ccd39'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.024162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb5ce192c764fe4946542f5960ccd39'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.037243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb5ce192c764fe4946542f5960ccd39'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.037973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb5ce192c764fe4946542f5960ccd39'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.045085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f0e3243e-914b-41a5-a894-448382bee31d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb5ce192c764fe4946542f5960ccd39'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.051926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1323/2645] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:54:59 2022] DELETE /identity/v3/users/cdb5ce192c764fe4946542f5960ccd39 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:00.231582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.242323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972667b-466d-4f9c-86fc-adb292dc9855 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.303950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.304568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.304846 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.305119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] PATH_INFO: `/v3/projects/500be47836b44a769f676e3e4bc00721` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.306422 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=500be47836b44a769f676e3e4bc00721)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.308253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.355241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500be47836b44a769f676e3e4bc00721', 'request_id': 'req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.361337 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500be47836b44a769f676e3e4bc00721', 'request_id': 'req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.367734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500be47836b44a769f676e3e4bc00721', 'request_id': 'req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.382412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] The token cache is being invalidate because project 500be47836b44a769f676e3e4bc00721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:00.382694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.383345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-f9bacb1e-0b70-41d8-ae79-99622d62e2d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:00.384890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1323/2646] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:00 2022] DELETE /identity/v3/projects/500be47836b44a769f676e3e4bc00721 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:55:00.392285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b949e-4368-4833-8b45-992997222e8c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.392597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b949e-4368-4833-8b45-992997222e8c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.392896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b949e-4368-4833-8b45-992997222e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.414327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b35b949e-4368-4833-8b45-992997222e8c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:00.415651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b35b949e-4368-4833-8b45-992997222e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:00.477668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1324/2647] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:00.485964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.486243 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.486669 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.512134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:00.513374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:00.525083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1324/2648] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:00.529411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b949e-4368-4833-8b45-992997222e8c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.542060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b949e-4368-4833-8b45-992997222e8c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.546558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.547181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.547455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.547925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.548741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:55:00.549042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.553669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.566085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1325/2649] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:55:00.570570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.580774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951c129-14db-4fdc-8ae9-da37fe7d61cf None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.586188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.586780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.587057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.587321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.587979 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.589879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.601755 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:00.630101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1325/2650] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:00 2022] POST /identity/v3/projects => generated 361 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:00.634461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.645144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404e7fb-4ef3-4c45-adbd-e298a0fbb208 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.705088 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.705696 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.705971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.706242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.706899 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.709016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.730338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1326/2651] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:00 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:00.734620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.745245 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc4738-9aad-4dc0-8234-e3b5f6591255 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.749752 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.750875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.751144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.751412 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.752077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.753796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.763449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1326/2652] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:00.768235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.778772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e24e78-b8ff-461a-8474-8fdb503aed1c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.784430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df29d41-83a5-4020-8f29-a1e1d6ca5853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.785389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29d41-83a5-4020-8f29-a1e1d6ca5853 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.785861 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29d41-83a5-4020-8f29-a1e1d6ca5853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.786420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29d41-83a5-4020-8f29-a1e1d6ca5853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.787575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df29d41-83a5-4020-8f29-a1e1d6ca5853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.790689 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df29d41-83a5-4020-8f29-a1e1d6ca5853 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.806319 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1327/2653] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:00.808473 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:00.819524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0ef78-6ff0-42c8-8ce1-2dc54701af0d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:00.824756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:00.825417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.825695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.825969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] PATH_INFO: `/v3/projects/ad8792b7b2c64459ab6a74d2cea4507c/users/d44a6a934f3b4ac486d9247c7727d71d/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.843441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8792b7b2c64459ab6a74d2cea4507c, user_id=d44a6a934f3b4ac486d9247c7727d71d, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:00.846431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:00.868117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1327/2654] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:00 2022] PUT /identity/v3/projects/ad8792b7b2c64459ab6a74d2cea4507c/users/d44a6a934f3b4ac486d9247c7727d71d/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:00.872552 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:00.873160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:00.873389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:00.904134 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] MFA Rules not processed for user `d44a6a934f3b4ac486d9247c7727d71d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:00.905353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:00.918149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:00.979548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1328/2655] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:00.985776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.065863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700a9c2-7062-49aa-a39f-c5854fa4bbe9 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.072052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.072672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.072948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.073216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.073861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.075791 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.103768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1328/2656] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:00 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:01.108081 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.121298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329924a-2663-441d-bf48-6ec4a50bd90c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.183284 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.183952 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.184218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.184493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.185160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.187079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.208063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1329/2657] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:01 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:55:01.212278 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.223135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37c94dc-5404-4c2e-af62-6ca457dd6fcc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.227831 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.228421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.228698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.228971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.229610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.231198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.243209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1329/2658] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:01.247809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.257923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1087b-40f9-430e-bfeb-d147f0a1f369 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.262845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.263605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.263929 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.264199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] PATH_INFO: `/v3/projects/cb060de10d43426887159cf61dac451d/users/d20596a78d0e40a58deae58a86279ce3/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.279810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb060de10d43426887159cf61dac451d, user_id=d20596a78d0e40a58deae58a86279ce3, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.283035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.296107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1330/2659] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:01 2022] PUT /identity/v3/projects/cb060de10d43426887159cf61dac451d/users/d20596a78d0e40a58deae58a86279ce3/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:55:01.300786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.311860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549536a-44a4-4d4b-9224-270bb1f1ecc1 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.369494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2a49-cd81-4c0f-8b85-48b8ab6010c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.370073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2a49-cd81-4c0f-8b85-48b8ab6010c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.370351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2a49-cd81-4c0f-8b85-48b8ab6010c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.370619 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2a49-cd81-4c0f-8b85-48b8ab6010c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.371267 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2a49-cd81-4c0f-8b85-48b8ab6010c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.373062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2a49-cd81-4c0f-8b85-48b8ab6010c5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.383378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1330/2660] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:01.387656 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.397944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3dd8f-9d32-4424-91b3-7efbbbda752d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.402492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.403108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.403378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.403643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] PATH_INFO: `/v3/projects/cb060de10d43426887159cf61dac451d/users/d20596a78d0e40a58deae58a86279ce3/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.405561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb060de10d43426887159cf61dac451d, user_id=d20596a78d0e40a58deae58a86279ce3, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.408718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.418684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1331/2661] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:01 2022] PUT /identity/v3/projects/cb060de10d43426887159cf61dac451d/users/d20596a78d0e40a58deae58a86279ce3/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:01.431068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.431438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.431814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.459632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] MFA Rules not processed for user `d20596a78d0e40a58deae58a86279ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:01.460870 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:01.474250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:01.533311 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1331/2662] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:01.544504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.546160 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:01.603266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b9c4f-0293-490f-bfd4-fd020dc71c91 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.606914 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.607515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.607867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.608094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.616963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.618668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.628216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1332/2663] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:01.701542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:01.711823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe9342-4582-4a72-8175-18757fd082f7 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:01.715270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185839a-2304-4c80-bf73-68a3bcd34f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:01.715895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185839a-2304-4c80-bf73-68a3bcd34f24 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:01.716169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185839a-2304-4c80-bf73-68a3bcd34f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:01.716435 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185839a-2304-4c80-bf73-68a3bcd34f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:01.771351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185839a-2304-4c80-bf73-68a3bcd34f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:01.772797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185839a-2304-4c80-bf73-68a3bcd34f24 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:01.782324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1332/2664] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:04.497162 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:04.507440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:04.561313 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95962f-b751-4b16-a254-bd3ebace347b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:04.566370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:04.567041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:04.567347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:04.567826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] PATH_INFO: `/v3/users/d44a6a934f3b4ac486d9247c7727d71d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:04.568950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44a6a934f3b4ac486d9247c7727d71d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:04.570862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:04.616810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44a6a934f3b4ac486d9247c7727d71d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:04.623273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44a6a934f3b4ac486d9247c7727d71d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:04.636090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44a6a934f3b4ac486d9247c7727d71d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:04.636735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44a6a934f3b4ac486d9247c7727d71d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:04.643488 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44a6a934f3b4ac486d9247c7727d71d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:04.649343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1333/2665] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:04 2022] DELETE /identity/v3/users/d44a6a934f3b4ac486d9247c7727d71d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:04.653914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:04.654414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:04.654414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:04.674017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:04.675230 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:04.733234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1333/2666] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:04.743868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:04.745178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:04.801799 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef72b32-4630-4182-9417-df57d1ff5cdf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:04.805547 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:04.806166 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:04.806426 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:04.806707 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:04.816399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:04.817844 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:04.827402 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1334/2667] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:05.092769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:05.094335 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:05.106381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43524cf3-fab0-46ab-bdbb-fbfdcd0aa8f2 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:05.111872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:05.112525 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.112792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.113063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] PATH_INFO: `/v3/projects/ad8792b7b2c64459ab6a74d2cea4507c` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.114190 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8792b7b2c64459ab6a74d2cea4507c)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:05.116113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:05.162639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:05.165845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8792b7b2c64459ab6a74d2cea4507c', 'request_id': 'req-a93da3e0-34ad-4b02-bfbf-575507963db8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.171944 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8792b7b2c64459ab6a74d2cea4507c', 'request_id': 'req-a93da3e0-34ad-4b02-bfbf-575507963db8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.178341 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8792b7b2c64459ab6a74d2cea4507c', 'request_id': 'req-a93da3e0-34ad-4b02-bfbf-575507963db8'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.190787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] The token cache is being invalidate because project ad8792b7b2c64459ab6a74d2cea4507c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:05.191073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.191694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.193215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1334/2668] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:05 2022] DELETE /identity/v3/projects/ad8792b7b2c64459ab6a74d2cea4507c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:05.197173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:05.207243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440cdbf5-6b62-462a-bfea-fa5ec4462160 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:05.238941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:05.270180 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:05.270804 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.271103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.271463 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] PATH_INFO: `/v3/users/d20596a78d0e40a58deae58a86279ce3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.272761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20596a78d0e40a58deae58a86279ce3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:05.274773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:05.320610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20596a78d0e40a58deae58a86279ce3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.327647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20596a78d0e40a58deae58a86279ce3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.339297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20596a78d0e40a58deae58a86279ce3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.339960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20596a78d0e40a58deae58a86279ce3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.346993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9f9885e7-de5c-43c7-935f-f3cf2e75c4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20596a78d0e40a58deae58a86279ce3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.352747 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1335/2669] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:05 2022] DELETE /identity/v3/users/d20596a78d0e40a58deae58a86279ce3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:05.630519 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:05.631992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:05.646418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93da3e0-34ad-4b02-bfbf-575507963db8 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:05.703529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:05.704219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.704368 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.704639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] PATH_INFO: `/v3/projects/cb060de10d43426887159cf61dac451d` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.705758 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb060de10d43426887159cf61dac451d)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:05.707647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:05.755069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb060de10d43426887159cf61dac451d', 'request_id': 'req-4ff5ead8-54a5-4680-81e6-acf6a8d72585'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.761572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb060de10d43426887159cf61dac451d', 'request_id': 'req-4ff5ead8-54a5-4680-81e6-acf6a8d72585'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.768342 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb060de10d43426887159cf61dac451d', 'request_id': 'req-4ff5ead8-54a5-4680-81e6-acf6a8d72585'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.781227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] The token cache is being invalidate because project cb060de10d43426887159cf61dac451d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:05.781534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.782212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-4ff5ead8-54a5-4680-81e6-acf6a8d72585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:05.783722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1335/2670] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:05 2022] DELETE /identity/v3/projects/cb060de10d43426887159cf61dac451d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:05.788698 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.789193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.789295 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.810321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:05.811482 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:05.870262 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1336/2671] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:05.878793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.879100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.879370 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.899475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:05.900710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:05.908630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1336/2672] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:05.912931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:05.922454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24afd0-7ffe-4119-a230-6e3d85ea299f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:05.927395 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:05.928067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.928287 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.928534 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.929168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:55:05.929504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:05.931058 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:05.943146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1337/2673] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:55:05.947380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:05.957285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3175496-1ed7-4d65-8dbb-07f6d3123c00 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:05.961871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:05.962464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:05.962742 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:05.963014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:05.963652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:05.965605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:05.993421 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1337/2674] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:05 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:05.997555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.008056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3433eb08-0655-455c-bb34-61f4de236ed3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.040265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:06.066277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.066895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.067170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.067443 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.068111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.070116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.092240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1338/2675] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:05 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:55:06.096429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.106743 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f9d99-18e2-47bb-bd17-bf16544bb437 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.111893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.112485 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.112793 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.113060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.113692 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.115262 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.127063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1338/2676] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:06.131146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.141285 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432202d-04b3-4b57-a455-f9d9284f36bc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.145921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a843c-7e26-454f-9cc3-483453098e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.146524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a843c-7e26-454f-9cc3-483453098e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.146803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a843c-7e26-454f-9cc3-483453098e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.147077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a843c-7e26-454f-9cc3-483453098e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.147713 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a843c-7e26-454f-9cc3-483453098e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.149348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a843c-7e26-454f-9cc3-483453098e1b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.159563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1339/2677] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:06.163735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.174170 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc388d-aff1-490a-8a2b-d7d38ca26438 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.179164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.179864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.180125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.180394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] PATH_INFO: `/v3/projects/a8f70e0aa9ab4f78a5b61413a884eb0a/users/3e7dce7db0f840d79823a79973e6904f/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.195403 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f70e0aa9ab4f78a5b61413a884eb0a, user_id=3e7dce7db0f840d79823a79973e6904f, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.198065 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.206175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:06.211072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1339/2678] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:06 2022] PUT /identity/v3/projects/a8f70e0aa9ab4f78a5b61413a884eb0a/users/3e7dce7db0f840d79823a79973e6904f/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:06.222398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.222673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.222938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.250109 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] MFA Rules not processed for user `3e7dce7db0f840d79823a79973e6904f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:06.251893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:06.313943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1340/2679] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:06.318995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.379390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9c6b2b-be3d-4123-8678-694bd7cad415 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.383991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.384592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.384868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.385136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.385757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.387639 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.414177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1340/2680] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:06 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:06.418281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.429324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7122a1bc-95c7-4f56-81ed-541570b395f2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.484100 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.485298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.485578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.485846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.486483 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.488422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.509456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1341/2681] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:06 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:06.513541 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.524306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bdfe5-f58a-47d6-9a35-28c3a4b7569f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.528920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.529516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.529787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.530052 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.530800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.532915 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.543343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1341/2682] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:06.548108 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.558063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d421e9-db98-450d-a8cb-78f6e2686a0f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.562653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.563294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.563570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.563883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] PATH_INFO: `/v3/projects/82e1b75a2308463d9a9acd1acda1b4b1/users/ea74268ed9224898888a860aca69bf96/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.578294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e1b75a2308463d9a9acd1acda1b4b1, user_id=ea74268ed9224898888a860aca69bf96, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.580992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.593765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1342/2683] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:06 2022] PUT /identity/v3/projects/82e1b75a2308463d9a9acd1acda1b4b1/users/ea74268ed9224898888a860aca69bf96/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:06.598299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.608227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3e94a-76e3-44ee-b63f-0cb0663c6081 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.664507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2c48e-5214-48a4-ba24-5f3d601df892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.665110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c48e-5214-48a4-ba24-5f3d601df892 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.665385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c48e-5214-48a4-ba24-5f3d601df892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.665648 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c48e-5214-48a4-ba24-5f3d601df892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.666270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c48e-5214-48a4-ba24-5f3d601df892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.667871 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c48e-5214-48a4-ba24-5f3d601df892 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.678114 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1342/2684] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:06.682390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.692472 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0cf59-865e-45ae-b44c-d6aa97b55fc4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.696964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.697594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.697864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.698146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] PATH_INFO: `/v3/projects/82e1b75a2308463d9a9acd1acda1b4b1/users/ea74268ed9224898888a860aca69bf96/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.700022 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e1b75a2308463d9a9acd1acda1b4b1, user_id=ea74268ed9224898888a860aca69bf96, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.702642 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.708294 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:06.714051 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1343/2685] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:06 2022] PUT /identity/v3/projects/82e1b75a2308463d9a9acd1acda1b4b1/users/ea74268ed9224898888a860aca69bf96/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:06.724757 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.725244 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.725345 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.753634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] MFA Rules not processed for user `ea74268ed9224898888a860aca69bf96`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:06.754766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:06.814281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:06.824334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1343/2686] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:06.835434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:06.837163 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:06.894738 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72130c-c372-4c94-bff2-827ace8cbca2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:06.898385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d856bb3-97b9-41a5-b190-9992ee172f78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:06.898984 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d856bb3-97b9-41a5-b190-9992ee172f78 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:06.899260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d856bb3-97b9-41a5-b190-9992ee172f78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:06.899529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d856bb3-97b9-41a5-b190-9992ee172f78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:06.909158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d856bb3-97b9-41a5-b190-9992ee172f78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:06.910901 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d856bb3-97b9-41a5-b190-9992ee172f78 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:06.920363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1344/2687] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:08.265666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:08.326346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfae90-3b95-4f66-85f9-ea60d891106a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:08.331203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:08.331858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:08.332134 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:08.332399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] PATH_INFO: `/v3/users/3e7dce7db0f840d79823a79973e6904f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:08.333497 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7dce7db0f840d79823a79973e6904f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:08.335380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:08.380738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7dce7db0f840d79823a79973e6904f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.387604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7dce7db0f840d79823a79973e6904f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.395121 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7dce7db0f840d79823a79973e6904f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.395789 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7dce7db0f840d79823a79973e6904f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.408092 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7dce7db0f840d79823a79973e6904f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.413890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1344/2688] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:08 2022] DELETE /identity/v3/users/3e7dce7db0f840d79823a79973e6904f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:08.418380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:08.418842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:08.418974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:08.437215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:08.438451 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:08.496595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1345/2689] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:08.506389 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:08.507840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:08.564596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6689a7-39d5-4ba8-bd5d-f48c912f9f78 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:08.568317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:08.568924 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:08.569202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:08.569478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:08.579305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:08.580734 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:08.590800 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1345/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:08.858852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:08.860397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:08.873944 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a147a-7c32-40e2-ae11-d228f96d894d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:08.879852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:08.880569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:08.880846 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:08.881115 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] PATH_INFO: `/v3/projects/a8f70e0aa9ab4f78a5b61413a884eb0a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:08.882320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f70e0aa9ab4f78a5b61413a884eb0a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:08.884209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:08.930750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f70e0aa9ab4f78a5b61413a884eb0a', 'request_id': 'req-0bcbd66e-e9a6-44e2-848a-e05c37886180'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.938223 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f70e0aa9ab4f78a5b61413a884eb0a', 'request_id': 'req-0bcbd66e-e9a6-44e2-848a-e05c37886180'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.943956 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f70e0aa9ab4f78a5b61413a884eb0a', 'request_id': 'req-0bcbd66e-e9a6-44e2-848a-e05c37886180'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.955320 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] The token cache is being invalidate because project a8f70e0aa9ab4f78a5b61413a884eb0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:08.955602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.956266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:08.957731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1346/2691] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:08 2022] DELETE /identity/v3/projects/a8f70e0aa9ab4f78a5b61413a884eb0a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:55:08.961748 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:08.971393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3662f-df78-4460-b67f-49ddb1d793c2 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:08.983989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:09.029623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.030208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.030480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.030745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] PATH_INFO: `/v3/users/ea74268ed9224898888a860aca69bf96` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.031902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea74268ed9224898888a860aca69bf96)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.033786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.080253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea74268ed9224898888a860aca69bf96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.088125 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea74268ed9224898888a860aca69bf96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.095456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea74268ed9224898888a860aca69bf96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.096120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea74268ed9224898888a860aca69bf96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.107723 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-107f20d2-7448-4f40-bcc7-b17040f6fcdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea74268ed9224898888a860aca69bf96'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.113625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1346/2692] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:08 2022] DELETE /identity/v3/users/ea74268ed9224898888a860aca69bf96 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:09.381131 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.382680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:09.397809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcbd66e-e9a6-44e2-848a-e05c37886180 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.420400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:09.459314 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.459992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.460264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.460527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] PATH_INFO: `/v3/projects/82e1b75a2308463d9a9acd1acda1b4b1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.461620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e1b75a2308463d9a9acd1acda1b4b1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.463537 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.510179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e1b75a2308463d9a9acd1acda1b4b1', 'request_id': 'req-2f16d9c8-ee75-428e-aa1e-fddee08aac68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.517779 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e1b75a2308463d9a9acd1acda1b4b1', 'request_id': 'req-2f16d9c8-ee75-428e-aa1e-fddee08aac68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.523297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e1b75a2308463d9a9acd1acda1b4b1', 'request_id': 'req-2f16d9c8-ee75-428e-aa1e-fddee08aac68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.534089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] The token cache is being invalidate because project 82e1b75a2308463d9a9acd1acda1b4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:09.534376 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.535008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f16d9c8-ee75-428e-aa1e-fddee08aac68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:09.536516 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1347/2693] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:09 2022] DELETE /identity/v3/projects/82e1b75a2308463d9a9acd1acda1b4b1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:09.541677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.542189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.542287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.563231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:09.564495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:09.624371 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1347/2694] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:09.632816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.633099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.633373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.653120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:09.654321 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:09.663237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1348/2695] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:09.667414 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.677096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005beda5-bceb-489e-86d5-57f5d46c2b5b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.681679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.682276 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.682550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.682817 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.683455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:55:09.683763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.685374 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.697208 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1348/2696] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:55:09.701418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.711044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583bb3c-124f-4fa1-b253-acafbe4c7eaa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.715655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.716275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.716549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.716817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.717436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.719353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.745810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1349/2697] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:09 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:09.749855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.762502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded9225-a657-4e4c-b32f-4dd25e28385a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.784253 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:09.824449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.825171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.825458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.825799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.826566 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.828469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.849802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1349/2698] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:09 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:09.853961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.864808 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78320da-e1cb-41b9-a0fa-af7aac8955b4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.869581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.870169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.870440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.870710 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.871392 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.873161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.884470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1350/2699] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:09.888623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.898578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca4cd9-e044-4ee8-8f7c-4e9ef092b0a3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.903287 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a580292-b052-46c8-8639-5b19931d07b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.903912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a580292-b052-46c8-8639-5b19931d07b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.904191 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a580292-b052-46c8-8639-5b19931d07b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.904464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a580292-b052-46c8-8639-5b19931d07b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.905100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a580292-b052-46c8-8639-5b19931d07b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.906686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a580292-b052-46c8-8639-5b19931d07b8 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.916874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1350/2700] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:09.921631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:09.934154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9d2bcb-dbdc-4238-a83c-6f0f53eb80dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:09.939241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:09.939919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.940191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.940456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] PATH_INFO: `/v3/projects/a0117453ce99423d91fc97df2a42c781/users/75778722ea5f432b971be9b4595fd4bc/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:09.954741 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0117453ce99423d91fc97df2a42c781, user_id=75778722ea5f432b971be9b4595fd4bc, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:09.957450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:09.969760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1351/2701] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:09 2022] PUT /identity/v3/projects/a0117453ce99423d91fc97df2a42c781/users/75778722ea5f432b971be9b4595fd4bc/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:09.981336 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:09.981613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:09.981879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.010142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] MFA Rules not processed for user `75778722ea5f432b971be9b4595fd4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:10.011325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:10.076801 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1351/2702] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:10.081825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.135327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:10.149015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a69acd-8710-4a2d-96e2-92241d77c9e6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.154407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.155015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.155288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.155563 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.156230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.158141 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.184681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1352/2703] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:10 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:55:10.189502 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.202312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5f5f4-4348-48d9-a793-353b94bb56e6 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.262483 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.263096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.263372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.263642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.264303 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.266825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.289078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1352/2704] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:10 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:10.293291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.307226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8babf702-0132-479d-8e24-389699f95dc7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.314145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.314145 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.314415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.314594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.315337 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.316855 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.328015 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1353/2705] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:10.332708 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.343160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbf35f-a827-4d29-ab64-fca2bbc172da admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.347878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.348534 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.348812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.349083 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] PATH_INFO: `/v3/projects/fe7f347f8c22416d89c51ea92a43f829/users/a2e140ae96e04b6891a961bbf98c34fe/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.364395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe7f347f8c22416d89c51ea92a43f829, user_id=a2e140ae96e04b6891a961bbf98c34fe, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.367023 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.376706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:10.382735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1353/2706] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:10 2022] PUT /identity/v3/projects/fe7f347f8c22416d89c51ea92a43f829/users/a2e140ae96e04b6891a961bbf98c34fe/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:10.387031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.398481 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b42f9-071c-4e92-b91f-793752b992dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.455415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1810e-b4b7-497d-ba6f-a2e9a088a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.456117 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1810e-b4b7-497d-ba6f-a2e9a088a7d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.456325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1810e-b4b7-497d-ba6f-a2e9a088a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.456595 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1810e-b4b7-497d-ba6f-a2e9a088a7d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.457246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1810e-b4b7-497d-ba6f-a2e9a088a7d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.458895 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1810e-b4b7-497d-ba6f-a2e9a088a7d3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.469181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1354/2707] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:10.473556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.485714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f36ca-f21e-45bf-be00-93d5397f0a3b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.490349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.491009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.491283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.491555 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] PATH_INFO: `/v3/projects/fe7f347f8c22416d89c51ea92a43f829/users/a2e140ae96e04b6891a961bbf98c34fe/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.493769 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe7f347f8c22416d89c51ea92a43f829, user_id=a2e140ae96e04b6891a961bbf98c34fe, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.496446 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.507701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1354/2708] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:10 2022] PUT /identity/v3/projects/fe7f347f8c22416d89c51ea92a43f829/users/a2e140ae96e04b6891a961bbf98c34fe/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:10.518669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.518953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.519226 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.548722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] MFA Rules not processed for user `a2e140ae96e04b6891a961bbf98c34fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:10.549926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:10.620104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1355/2709] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:10.633674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:10.635469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:10.694859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7791-e381-4528-8568-7a1a7148feee admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:10.698482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d0bdf-9f82-4423-be0e-c99b8a320a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:10.699088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d0bdf-9f82-4423-be0e-c99b8a320a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:10.699357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d0bdf-9f82-4423-be0e-c99b8a320a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:10.699624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d0bdf-9f82-4423-be0e-c99b8a320a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:10.708441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d0bdf-9f82-4423-be0e-c99b8a320a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:10.709879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d0bdf-9f82-4423-be0e-c99b8a320a2f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:10.719756 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1355/2710] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:11.292044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:11.339118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:11.354703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5104c-de60-4500-b7c1-a23c30afdc60 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:11.359239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:11.360390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:11.360666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:11.360935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] PATH_INFO: `/v3/users/75778722ea5f432b971be9b4595fd4bc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:11.362042 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75778722ea5f432b971be9b4595fd4bc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:11.364031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:11.410503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75778722ea5f432b971be9b4595fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.417782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75778722ea5f432b971be9b4595fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.430288 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75778722ea5f432b971be9b4595fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.431018 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75778722ea5f432b971be9b4595fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.438493 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75778722ea5f432b971be9b4595fd4bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.444328 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1356/2711] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:11 2022] DELETE /identity/v3/users/75778722ea5f432b971be9b4595fd4bc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:55:11.448879 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:11.449162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:11.449457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:11.468189 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:11.469405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:11.529100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1356/2712] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:11.539124 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:11.540569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:11.596918 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a629b1b-af8c-4249-801d-9c8747c78864 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:11.600631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:11.601237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:11.601514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:11.601787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:11.611444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:11.612915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:11.622584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1357/2713] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:55:11.893826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:11.895297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:11.907761 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fa127-2614-4586-82ec-1733cb83a5dd None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:11.913031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:11.913675 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:11.913950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:11.914263 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] PATH_INFO: `/v3/projects/a0117453ce99423d91fc97df2a42c781` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:11.915339 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0117453ce99423d91fc97df2a42c781)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:11.917367 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:11.966206 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0117453ce99423d91fc97df2a42c781', 'request_id': 'req-88cf4a38-1540-4225-96ef-66b65f88bcfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.971760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0117453ce99423d91fc97df2a42c781', 'request_id': 'req-88cf4a38-1540-4225-96ef-66b65f88bcfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.978221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0117453ce99423d91fc97df2a42c781', 'request_id': 'req-88cf4a38-1540-4225-96ef-66b65f88bcfb'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.990720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] The token cache is being invalidate because project a0117453ce99423d91fc97df2a42c781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:11.991003 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.991625 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:11.993155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1357/2714] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:11 2022] DELETE /identity/v3/projects/a0117453ce99423d91fc97df2a42c781 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:11.997427 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.007270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca9852-85e2-4837-a6b1-8b997f2642fd service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.061970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.062514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.063356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.063632 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] PATH_INFO: `/v3/users/a2e140ae96e04b6891a961bbf98c34fe` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.064824 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e140ae96e04b6891a961bbf98c34fe)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.066757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.077673 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:12.114030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e140ae96e04b6891a961bbf98c34fe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.121214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e140ae96e04b6891a961bbf98c34fe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.133803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e140ae96e04b6891a961bbf98c34fe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.134494 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e140ae96e04b6891a961bbf98c34fe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.141399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7c99f954-e1f8-4474-a990-01c0320cecdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e140ae96e04b6891a961bbf98c34fe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.147220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1358/2715] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:11 2022] DELETE /identity/v3/users/a2e140ae96e04b6891a961bbf98c34fe => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:55:12.424659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.426162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:12.441703 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf4a38-1540-4225-96ef-66b65f88bcfb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.450610 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:12.503906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.504544 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.504820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.505087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] PATH_INFO: `/v3/projects/fe7f347f8c22416d89c51ea92a43f829` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.506199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe7f347f8c22416d89c51ea92a43f829)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.508122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.553856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe7f347f8c22416d89c51ea92a43f829', 'request_id': 'req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.559297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe7f347f8c22416d89c51ea92a43f829', 'request_id': 'req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.565819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe7f347f8c22416d89c51ea92a43f829', 'request_id': 'req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.579489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] The token cache is being invalidate because project fe7f347f8c22416d89c51ea92a43f829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:12.579802 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.580430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-fab3c6ce-7c0c-4f5e-914a-2d02eea060e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:12.581967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1358/2716] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:12 2022] DELETE /identity/v3/projects/fe7f347f8c22416d89c51ea92a43f829 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:12.587253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.587767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.587870 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.606997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:12.608164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:12.665743 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1359/2717] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:12.674849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.675128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.675400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.695250 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:12.696494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:12.705060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1359/2718] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:12.709372 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.718960 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07ce8-6b09-4cf8-bedc-d002f9996bbf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.723454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.724077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.724351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.724617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.725254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:55:12.725558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.727140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.738503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1360/2719] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:55:12.742916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.752818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eac3d5-0082-402a-b7ad-3a70407db9ee None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.757420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.758008 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.758284 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.758550 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.759164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.761169 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.787588 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1360/2720] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:12 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:12.791692 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.802230 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72d2d3-417c-4da5-b8b8-597dca2fb788 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.859173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.860076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.860354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.860626 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.861247 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.863165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.884327 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1361/2721] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:12 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:12.888515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.899167 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e27b-b05b-4a78-a9d0-0ca685a12cb3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.903682 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.904304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.904578 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.904843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.905475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.907620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.917554 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1361/2722] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:12.921362 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.927876 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:12.931940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e73043-a89e-4ee5-bf38-16922eeaabb6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.937652 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54840a-c905-4238-b7be-5f063caeb635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.938260 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54840a-c905-4238-b7be-5f063caeb635 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.938577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54840a-c905-4238-b7be-5f063caeb635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.939038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54840a-c905-4238-b7be-5f063caeb635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.939695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54840a-c905-4238-b7be-5f063caeb635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.941381 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54840a-c905-4238-b7be-5f063caeb635 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:12.951907 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1362/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:12.956568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:12.966531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eb32c-5772-420a-86dc-4ea64cf85e36 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:12.971955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:12.972621 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:12.972896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:12.973172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] PATH_INFO: `/v3/projects/392f2f43f0a04a1a9e7bb016a922eca9/users/243dd1d9dd8a490a90cf20c301fbd2f4/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:12.988034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392f2f43f0a04a1a9e7bb016a922eca9, user_id=243dd1d9dd8a490a90cf20c301fbd2f4, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:12.990690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.003667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1362/2724] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:12 2022] PUT /identity/v3/projects/392f2f43f0a04a1a9e7bb016a922eca9/users/243dd1d9dd8a490a90cf20c301fbd2f4/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:13.013842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.014118 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.014389 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.041888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] MFA Rules not processed for user `243dd1d9dd8a490a90cf20c301fbd2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:13.043008 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:13.108450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1363/2725] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:13.113042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.155214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:13.176433 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef0383-99ab-4118-9aca-709c28813c69 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.180960 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.181553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.181823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.182088 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.182713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.184720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.210663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1363/2726] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:13 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:13.214825 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.225323 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584d62d-d02e-4fe9-9ef2-8738c0381640 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.281255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.282044 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.282374 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.282681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.283322 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.285290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.306152 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1364/2727] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:13 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:13.310305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.321128 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ce0de-096d-4ad5-97d0-0e3ccf82b07d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.325874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.326409 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.326684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.326955 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.327593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.329823 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.340364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1364/2728] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:13.344618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.354555 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420d85a-57f7-47a5-8d63-edf58248008b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.359140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.359830 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.360110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.360383 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] PATH_INFO: `/v3/projects/79bc17772cd24da08cd378502ccc2b92/users/f0cce3a6c83241af8f3a83aad3ba72ea/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.374964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bc17772cd24da08cd378502ccc2b92, user_id=f0cce3a6c83241af8f3a83aad3ba72ea, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.378142 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.390599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1365/2729] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:13 2022] PUT /identity/v3/projects/79bc17772cd24da08cd378502ccc2b92/users/f0cce3a6c83241af8f3a83aad3ba72ea/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:13.395165 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.405070 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8648d-4861-4008-810a-79550ba7d4fb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.459672 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bc0c5-7571-4e34-b70c-64f873b15b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.460293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bc0c5-7571-4e34-b70c-64f873b15b65 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.460570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bc0c5-7571-4e34-b70c-64f873b15b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.460832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bc0c5-7571-4e34-b70c-64f873b15b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.461451 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bc0c5-7571-4e34-b70c-64f873b15b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.463006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bc0c5-7571-4e34-b70c-64f873b15b65 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.472560 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1365/2730] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:13 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:13.477514 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.487991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1d70d-b320-4775-a30e-52f4c5778f14 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.492533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.493174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.493810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.494060 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] PATH_INFO: `/v3/projects/79bc17772cd24da08cd378502ccc2b92/users/f0cce3a6c83241af8f3a83aad3ba72ea/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.496078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bc17772cd24da08cd378502ccc2b92, user_id=f0cce3a6c83241af8f3a83aad3ba72ea, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.498669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.509406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1366/2731] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:13 2022] PUT /identity/v3/projects/79bc17772cd24da08cd378502ccc2b92/users/f0cce3a6c83241af8f3a83aad3ba72ea/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:13.520306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.520587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.520858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.548237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] MFA Rules not processed for user `f0cce3a6c83241af8f3a83aad3ba72ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:13.549429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:13.619613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1366/2732] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:13.630811 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:13.632503 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:13.673289 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:13.691641 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace547c4-20c2-4fb7-b40e-5e1f730ab950 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:13.695624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:13.696267 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:13.696541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:13.696806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:13.752385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:13.753853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:13.762623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1367/2733] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:17.980334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:17.981690 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:17.993874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53f016-80eb-4781-8c83-f940478beb35 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:17.997397 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396e7f1-e6a1-45f1-bf00-0511efd43bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:17.997998 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e7f1-e6a1-45f1-bf00-0511efd43bbf service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:17.998272 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e7f1-e6a1-45f1-bf00-0511efd43bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:17.998542 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396e7f1-e6a1-45f1-bf00-0511efd43bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:18.007949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396e7f1-e6a1-45f1-bf00-0511efd43bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:18.009355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396e7f1-e6a1-45f1-bf00-0511efd43bbf service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:18.019463 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1367/2734] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:22.908582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:22.975181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e50ec-2291-4a09-b696-06c9e44a8824 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:22.979883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:22.980512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:22.980782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:22.981048 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] PATH_INFO: `/v3/users/243dd1d9dd8a490a90cf20c301fbd2f4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:22.982143 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] RBAC: Authorizing `identity:delete_user(user_id=243dd1d9dd8a490a90cf20c301fbd2f4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:22.984097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:23.031324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243dd1d9dd8a490a90cf20c301fbd2f4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.038620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243dd1d9dd8a490a90cf20c301fbd2f4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.052341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243dd1d9dd8a490a90cf20c301fbd2f4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.053052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243dd1d9dd8a490a90cf20c301fbd2f4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.060003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243dd1d9dd8a490a90cf20c301fbd2f4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.066701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1368/2735] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:22 2022] DELETE /identity/v3/users/243dd1d9dd8a490a90cf20c301fbd2f4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:23.071242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:23.071696 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:23.071828 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:23.091104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:23.092430 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:23.105026 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:23.153413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1368/2736] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:55:23.163325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:23.164782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:23.176215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:23.222760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7e9d3-9424-47a2-92f2-9055be91d09c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:23.226334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:23.226935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:23.227209 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:23.227478 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:23.236760 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:23.238205 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:23.248187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1369/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:23.513465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:23.514921 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:23.527095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae35a3-887d-4fc0-b5c1-5efc47005253 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:23.531704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:23.532535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:23.533369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:23.533662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] PATH_INFO: `/v3/projects/392f2f43f0a04a1a9e7bb016a922eca9` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:23.535104 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392f2f43f0a04a1a9e7bb016a922eca9)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:23.537103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:23.584068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392f2f43f0a04a1a9e7bb016a922eca9', 'request_id': 'req-6b04a55e-62d6-4207-9a71-66d7d565c019'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.589851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392f2f43f0a04a1a9e7bb016a922eca9', 'request_id': 'req-6b04a55e-62d6-4207-9a71-66d7d565c019'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.596878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392f2f43f0a04a1a9e7bb016a922eca9', 'request_id': 'req-6b04a55e-62d6-4207-9a71-66d7d565c019'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.608712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] The token cache is being invalidate because project 392f2f43f0a04a1a9e7bb016a922eca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:23.609017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.609596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.611649 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1369/2738] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:23 2022] DELETE /identity/v3/projects/392f2f43f0a04a1a9e7bb016a922eca9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:23.615631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:23.625255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1c30b-11ff-4141-88b7-347ccfb2b683 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:23.679379 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:23.680024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:23.680324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:23.680591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] PATH_INFO: `/v3/users/f0cce3a6c83241af8f3a83aad3ba72ea` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:23.681678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cce3a6c83241af8f3a83aad3ba72ea)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:23.683581 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:23.728416 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cce3a6c83241af8f3a83aad3ba72ea'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.735504 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cce3a6c83241af8f3a83aad3ba72ea'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.747408 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cce3a6c83241af8f3a83aad3ba72ea'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.748041 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cce3a6c83241af8f3a83aad3ba72ea'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.754809 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-76788cde-ebc0-4e25-b72c-74639b18bcb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cce3a6c83241af8f3a83aad3ba72ea'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:23.760630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1370/2739] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:23 2022] DELETE /identity/v3/users/f0cce3a6c83241af8f3a83aad3ba72ea => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:24.027904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.029327 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:24.042094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04a55e-62d6-4207-9a71-66d7d565c019 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.099647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.100375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.100574 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.100840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] PATH_INFO: `/v3/projects/79bc17772cd24da08cd378502ccc2b92` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.102011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bc17772cd24da08cd378502ccc2b92)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.103861 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.147890 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:24.150839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bc17772cd24da08cd378502ccc2b92', 'request_id': 'req-7934ce55-0bfb-4544-92d4-253a6c86f9e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:24.157294 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bc17772cd24da08cd378502ccc2b92', 'request_id': 'req-7934ce55-0bfb-4544-92d4-253a6c86f9e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:24.163775 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bc17772cd24da08cd378502ccc2b92', 'request_id': 'req-7934ce55-0bfb-4544-92d4-253a6c86f9e1'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:24.181139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] The token cache is being invalidate because project 79bc17772cd24da08cd378502ccc2b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:24.181427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:24.182068 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-7934ce55-0bfb-4544-92d4-253a6c86f9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:24.183613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1370/2740] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:24 2022] DELETE /identity/v3/projects/79bc17772cd24da08cd378502ccc2b92 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:24.190169 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.190637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.190757 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.210974 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:24.212208 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:24.238174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:24.273452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1371/2741] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:24.282480 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.283296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.283572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.303847 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:24.305051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:24.312874 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1371/2742] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:24.317265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.326912 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265cdd-a9e7-4793-92f4-82d6c53ced37 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.331499 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.332164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.332442 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.332711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.333348 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:55:24.333645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.335250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.347161 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1372/2743] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 28 09:55:24.351583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.360840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c430607-3969-4309-b68e-bc5a6f87e2cc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.366082 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.366677 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.366949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.367215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.367868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.369987 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.405613 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1372/2744] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:24 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:24.410391 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.420405 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8ca6ac-a978-4b9e-bbb0-3bb3d8a7ecf3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.477301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.477756 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.478035 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.478309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.478935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.480904 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.502842 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1373/2745] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:24 2022] POST /identity/v3/users => generated 281 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:24.507035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.517478 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76882325-26b3-4b33-915f-cd7d766ba24b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.522234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.522841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.523116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.523384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.524069 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.525646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.536122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1373/2746] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:24.540250 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.550672 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b876d0b-ccd8-470c-a490-b3dcafbec260 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.555263 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb1b27-2939-4a9f-8fda-17167d244598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.555896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb1b27-2939-4a9f-8fda-17167d244598 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.556173 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb1b27-2939-4a9f-8fda-17167d244598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.556445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb1b27-2939-4a9f-8fda-17167d244598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.557079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb1b27-2939-4a9f-8fda-17167d244598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.558678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebb1b27-2939-4a9f-8fda-17167d244598 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.569343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1374/2747] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:24.573632 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.584034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03c579-dc8b-4391-96ee-71ac992cdd4a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.591071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.591722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.592034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.592306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] PATH_INFO: `/v3/projects/f42dddcdf170456aaa0deecd71acc2b5/users/f831f2b9045c455c81db6e8bc2e56435/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.607427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42dddcdf170456aaa0deecd71acc2b5, user_id=f831f2b9045c455c81db6e8bc2e56435, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.610471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.624779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1374/2748] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:24 2022] PUT /identity/v3/projects/f42dddcdf170456aaa0deecd71acc2b5/users/f831f2b9045c455c81db6e8bc2e56435/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:24.637598 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.637817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.638085 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.666045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] MFA Rules not processed for user `f831f2b9045c455c81db6e8bc2e56435`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:24.667224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:24.731943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1375/2749] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:55:24.766760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.768688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:24.825976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51089b54-8be8-4030-b4d7-56f02a3ff357 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.830388 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9467c89-d680-4c4d-b651-7f1d8fcc5bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.831007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9467c89-d680-4c4d-b651-7f1d8fcc5bbe service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.831293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9467c89-d680-4c4d-b651-7f1d8fcc5bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.831551 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9467c89-d680-4c4d-b651-7f1d8fcc5bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.840515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9467c89-d680-4c4d-b651-7f1d8fcc5bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.841953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9467c89-d680-4c4d-b651-7f1d8fcc5bbe service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:24.850997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1375/2750] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:24.875398 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:24.905950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:24.950441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d0dd7-e0bf-4e01-850e-c4d038b5dbce None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:24.955644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:24.956298 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:24.956574 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:24.956843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] PATH_INFO: `/v3/users/f831f2b9045c455c81db6e8bc2e56435` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:24.957961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f831f2b9045c455c81db6e8bc2e56435)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:24.959905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.005634 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f831f2b9045c455c81db6e8bc2e56435'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.012549 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f831f2b9045c455c81db6e8bc2e56435'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.024852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f831f2b9045c455c81db6e8bc2e56435'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.025505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f831f2b9045c455c81db6e8bc2e56435'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.032199 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f831f2b9045c455c81db6e8bc2e56435'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.037942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1376/2751] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:55:24 2022] DELETE /identity/v3/users/f831f2b9045c455c81db6e8bc2e56435 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:25.042467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.042750 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.043019 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.061896 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:25.063098 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:25.122161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1376/2752] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:25.132545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.134056 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:25.191300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723c257-041d-4145-b845-2fed30241b2c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.194928 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daad059-e9c0-49c9-a755-2dd17b83f8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.195520 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad059-e9c0-49c9-a755-2dd17b83f8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.195823 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad059-e9c0-49c9-a755-2dd17b83f8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.196090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daad059-e9c0-49c9-a755-2dd17b83f8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.205153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad059-e9c0-49c9-a755-2dd17b83f8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.206584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daad059-e9c0-49c9-a755-2dd17b83f8f1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.215839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1377/2753] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:25.481139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.482689 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:25.491895 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:25.496175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648019d2-2b2d-4bb2-bfc5-d0353ac5fe7b None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.503132 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.504953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.505255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.506116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] PATH_INFO: `/v3/projects/f42dddcdf170456aaa0deecd71acc2b5` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.507387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42dddcdf170456aaa0deecd71acc2b5)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.509536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.561479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42dddcdf170456aaa0deecd71acc2b5', 'request_id': 'req-efad5008-42a0-43bd-8b43-e3933c688660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.568590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42dddcdf170456aaa0deecd71acc2b5', 'request_id': 'req-efad5008-42a0-43bd-8b43-e3933c688660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.575265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42dddcdf170456aaa0deecd71acc2b5', 'request_id': 'req-efad5008-42a0-43bd-8b43-e3933c688660'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.588194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] The token cache is being invalidate because project f42dddcdf170456aaa0deecd71acc2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:55:25.588476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.589076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-efad5008-42a0-43bd-8b43-e3933c688660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:55:25.590562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1377/2754] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:55:25 2022] DELETE /identity/v3/projects/f42dddcdf170456aaa0deecd71acc2b5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:25.596421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.596703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.596972 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.617021 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:25.618216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:25.663768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:25.685140 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1378/2755] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:25.693036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.693395 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.693699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.713564 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:25.714773 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:25.723375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1378/2756] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:25.727657 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.737890 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb49e63-6f54-4c74-a520-962e347ef9ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.742546 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.743154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.743433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.743996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.744816 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:55:25.745119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.746723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.758220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1379/2757] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:55:25.762695 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.772009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cefbf-3ce5-40be-8c08-6dd9bc1f5519 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.777443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.778047 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.778320 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.778579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.779214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.781122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.816035 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1379/2758] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:25 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:25.817164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.827423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39260898-5e77-4683-b4c3-03bc9a17e42a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.884011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.884621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.884897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.885164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.885802 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.887726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.910074 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1380/2759] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:25 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:55:25.914214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.924710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238e26-6b56-4e28-8bfd-ef25a50bd524 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.930634 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.931251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.931529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.931851 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.932527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.934120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.944406 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1380/2760] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:25.948452 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.958852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca7a93-0df7-480f-aa1e-e732f8f97721 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.963525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47017ed7-0e5c-4a60-bf1a-79e7ce97a63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.964184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47017ed7-0e5c-4a60-bf1a-79e7ce97a63c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.964448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47017ed7-0e5c-4a60-bf1a-79e7ce97a63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:25.964717 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47017ed7-0e5c-4a60-bf1a-79e7ce97a63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:25.965468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47017ed7-0e5c-4a60-bf1a-79e7ce97a63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:25.966964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47017ed7-0e5c-4a60-bf1a-79e7ce97a63c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:25.978399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1381/2761] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:55:25.982512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:25.993393 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08549ba1-e8c2-4022-b7cb-9a22dabc3115 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:25.998805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:25.999509 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:25.999819 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:26.000162 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] PATH_INFO: `/v3/projects/1659fef961ae4ccda1adf6b62c018c9e/users/c2771d02bec4465d94e98134015b9705/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:26.015717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1659fef961ae4ccda1adf6b62c018c9e, user_id=c2771d02bec4465d94e98134015b9705, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:26.018969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:26.032849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1381/2762] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:25 2022] PUT /identity/v3/projects/1659fef961ae4ccda1adf6b62c018c9e/users/c2771d02bec4465d94e98134015b9705/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:26.038373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b02478-0633-46be-83b2-d828cedd5c8b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:26.038650 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b02478-0633-46be-83b2-d828cedd5c8b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:26.038935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b02478-0633-46be-83b2-d828cedd5c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:26.057948 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-46b02478-0633-46be-83b2-d828cedd5c8b None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:26.059154 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-46b02478-0633-46be-83b2-d828cedd5c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:26.121533 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1382/2763] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:26.132725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:26.134479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:26.192681 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c0405-0ae6-444a-b4d3-1fe14a982e63 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:26.196229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:26.196827 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:26.197101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:26.197369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:26.207548 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:26.209060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:26.218351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1382/2764] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:30.152227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.152643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.152767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.183116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] MFA Rules not processed for user `c2771d02bec4465d94e98134015b9705`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:30.184368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:30.234196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:30.256746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1383/2765] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:30.261536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:30.272247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1634f-0b3c-4dfd-8287-90ba1b308f7e service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:30.277007 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:30.277603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.277883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.278159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.278824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:30.280765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:30.309184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1383/2766] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:55:30 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:55:30.313244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:30.325388 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535c343-94fc-43f5-b9f6-68fada299342 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:30.408939 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:30.409591 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.409863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.410125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.410746 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:30.412726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:30.435253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1384/2767] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:55:30 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:55:30.439126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:30.447663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:30.451671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa2823-d63e-47f3-a460-25f6874f9a99 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:30.457910 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:30.458738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.459182 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.459562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.460447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:30.462042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:30.476150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1384/2768] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:55:30.481401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:30.492688 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c81f-9bc9-4fb1-b20e-955b9c578891 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:30.497993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:30.498803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.499155 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.499585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] PATH_INFO: `/v3/projects/80c731d304704ac38498506b9f6a2b20/users/a6a08ed206484313bdab63aa8e6aaa20/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.516803 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c731d304704ac38498506b9f6a2b20, user_id=a6a08ed206484313bdab63aa8e6aaa20, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:30.519643 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:30.533704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1385/2769] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:30 2022] PUT /identity/v3/projects/80c731d304704ac38498506b9f6a2b20/users/a6a08ed206484313bdab63aa8e6aaa20/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:30.537826 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:30.548056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e9456a-d498-4384-955e-028e5f551595 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:30.606698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd13c8a-223c-43fb-9bc2-679132b4997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:30.607242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd13c8a-223c-43fb-9bc2-679132b4997e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.607516 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd13c8a-223c-43fb-9bc2-679132b4997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.607812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd13c8a-223c-43fb-9bc2-679132b4997e admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.608461 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd13c8a-223c-43fb-9bc2-679132b4997e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:30.610093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd13c8a-223c-43fb-9bc2-679132b4997e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:30.621858 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1385/2770] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:55:30.626084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:30.637112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcde14-7202-4996-91bb-609beee71d64 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:30.641900 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:30.642601 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:30.642873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:30.643139 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] PATH_INFO: `/v3/projects/80c731d304704ac38498506b9f6a2b20/users/a6a08ed206484313bdab63aa8e6aaa20/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:30.645083 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c731d304704ac38498506b9f6a2b20, user_id=a6a08ed206484313bdab63aa8e6aaa20, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:30.647715 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:30.658726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1386/2771] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:55:30 2022] PUT /identity/v3/projects/80c731d304704ac38498506b9f6a2b20/users/a6a08ed206484313bdab63aa8e6aaa20/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:55:34.933843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee108547-7bce-4b68-a8a8-1f75669bab1f None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:34.934476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee108547-7bce-4b68-a8a8-1f75669bab1f None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:34.934476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee108547-7bce-4b68-a8a8-1f75669bab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:34.968663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-ee108547-7bce-4b68-a8a8-1f75669bab1f None None] MFA Rules not processed for user `a6a08ed206484313bdab63aa8e6aaa20`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:34.970878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ee108547-7bce-4b68-a8a8-1f75669bab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:35.045161 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1386/2772] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:35.058753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:35.060455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:35.119378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712a065-e6d6-4bb0-85ae-537e37343aa2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:35.123014 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db830-6974-465f-ab64-7d02dded453a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:35.123623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db830-6974-465f-ab64-7d02dded453a service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:35.123941 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db830-6974-465f-ab64-7d02dded453a service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:35.124214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663db830-6974-465f-ab64-7d02dded453a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:35.190479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:35.201103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db830-6974-465f-ab64-7d02dded453a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:35.202666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663db830-6974-465f-ab64-7d02dded453a service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:35.214622 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1387/2773] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:35.393192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e521a0-30e1-4c92-b8c6-879afba6f012 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:35.393192 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e521a0-30e1-4c92-b8c6-879afba6f012 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:35.393536 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e521a0-30e1-4c92-b8c6-879afba6f012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:35.415647 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6e521a0-30e1-4c92-b8c6-879afba6f012 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:35.417226 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6e521a0-30e1-4c92-b8c6-879afba6f012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:35.478728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1387/2774] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:55:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:35.531214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db830-6974-465f-ab64-7d02dded453a service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:35.532384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-663db830-6974-465f-ab64-7d02dded453a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:35.628418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663db830-6974-465f-ab64-7d02dded453a service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:35.633390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:35.634113 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:35.634440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:35.634776 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:35.644599 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:35.646299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:35.655120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1388/2775] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:55:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:55:44.389950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936596-9e81-4331-ba4b-451db926417c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:44.390467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936596-9e81-4331-ba4b-451db926417c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:44.390467 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936596-9e81-4331-ba4b-451db926417c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:44.410101 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d6936596-9e81-4331-ba4b-451db926417c None None] MFA Rules not processed for user `c2771d02bec4465d94e98134015b9705`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:44.411235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d6936596-9e81-4331-ba4b-451db926417c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:44.418298 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1388/2776] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:55:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:44.459916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:44.469775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d94a9f-fab3-414e-8d97-a7c604c66a6f service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:44.474110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '416d5cf4e87c4ce9928ca7b4f0ee8220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:44.474700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:44.474971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:44.475237 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:44.477386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:44.547894 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:44.549961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72c378-fc1b-4273-ac9d-d3758b2e65f0 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:44.561004 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1389/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:45.104004 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6936596-9e81-4331-ba4b-451db926417c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:45.117537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:45.165829 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6936596-9e81-4331-ba4b-451db926417c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:45.169538 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:45.170984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:45.171372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:45.171738 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:45.174073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:45.189848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:45.191291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:45.203222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1389/2778] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:55:46.498439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:46.498967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:46.498967 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:46.555047 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:55:46.556431 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:46.576936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1390/2779] 10.222.0.52 () {58 vars in 1009 bytes} [Fri Oct 28 09:55:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:55:46.593565 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:46.614280 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72555b71-8604-48f1-9aee-a71cbbffdd18 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:46.620210 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:46.620824 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:46.621108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:46.622764 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:46.633028 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:46.634235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:46.643481 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1390/2780] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:55:56.573346 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:56.639381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=30 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:55:58.401469 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:55:58.402863 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:58.415158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa16fa-39d8-4ab1-b59a-25c7290a28f9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:55:58.418649 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:55:58.419236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:55:58.419506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:55:58.419797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:55:58.421455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:55:58.430702 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:55:58.432216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf2399-6482-4d52-b103-43ade40f39c3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:55:58.440538 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1391/2781] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:55:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:56:08.436631 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:56:09.375270 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:56:09.377624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:56:09.395386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea02328-1af6-4405-9e62-386c1efec706 service swift] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:56:09.409860 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:56:09.410563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:56:09.410843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:56:09.411113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:56:09.413349 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:56:09.432218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:56:09.434221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:56:09.447174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1391/2782] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:56:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:56:12.635020 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:56:12.635527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:56:12.635527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:56:12.656730 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] MFA Rules not processed for user `d38ce75423b5412d90c1158396978b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:56:12.657955 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:56:12.715975 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1392/2783] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:56:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:56:13.343075 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:56:13.344635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:56:13.420355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb1dd7-0bd4-40db-8067-0ceceb8889c6 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:56:13.425905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:56:13.426508 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:56:13.426782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:56:13.427050 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:56:13.428836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:56:13.440880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:56:13.442436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:56:13.452266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1392/2784] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:56:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:56:14.010664 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:56:14.021425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07bfff-d449-4e43-9a3f-1890b376c992 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:56:14.025185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:56:14.025819 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:56:14.026098 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:56:14.026366 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:56:14.037290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:56:14.038845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:56:14.049078 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1393/2785] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:56:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:56:23.294608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:56:23.308304 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac30b8-7286-438c-bc75-292672951ebb service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:56:23.313961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:56:23.314572 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:56:23.314841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:56:23.315108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:56:23.316884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:56:23.329234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:56:23.330842 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8f5a9-d19e-4767-87ea-5c4919f4aa6d service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:56:23.341903 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1393/2786] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:56:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:56:24.044225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=31 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:56:33.337055 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=44 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:57:42.678277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:57:42.690446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5ac41-7ed2-47ed-8752-88d0fe6efdb7 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:57:42.695695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:57:42.696338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:57:42.696608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:57:42.696873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:57:42.698572 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:57:42.709185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:57:42.710740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:57:42.720090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1394/2787] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:57:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:57:52.716093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:57:53.544587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:57:53.545146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:57:53.545146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:57:53.564605 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] MFA Rules not processed for user `d38ce75423b5412d90c1158396978b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:57:53.565832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:57:53.572770 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1394/2788] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:57:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:57:53.874487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:57:53.883858 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b16ec0-904b-43a4-b28c-f86742ba77b7 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:57:53.887462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:57:53.888079 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:57:53.888354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:57:53.888618 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:57:53.890936 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:57:53.900290 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:57:53.901711 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:57:53.910179 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1395/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:57:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:58:03.559266 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:58:03.906733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:58:07.129331 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:07.134850 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:07.151034 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dda8fa-eddc-4cf6-a142-2831381ad2b0 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:07.157124 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:07.157746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:07.158015 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:07.158281 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:07.160017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:07.173593 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:07.175184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e973e93-7c76-4112-8bb8-a8c8aee1c73b service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:07.186813 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1395/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:58:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:58:17.181670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:58:46.777368 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:46.789216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d146a7-d317-4276-8212-d5a368da168d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:46.794425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:46.795026 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:46.795305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:46.795575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:46.797352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:46.809037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:46.810561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:46.820935 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1396/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:58:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:58:47.733462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:47.733946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:47.734043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:47.752663 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] MFA Rules not processed for user `72d842d8875d4f3f8d44875c410ebf37`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:58:47.753835 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:47.761152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1396/2792] 10.222.0.52 () {58 vars in 967 bytes} [Fri Oct 28 09:58:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:58:47.801061 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:47.812276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e0a3-dec9-46f3-a67b-05a6a607b5c7 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:47.817184 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0cb99f8c316545e2ba685c87006cbfca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:47.817766 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:47.818040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:47.818310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:47.826191 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:47.827792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:47.836341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1397/2793] 10.222.0.52 () {60 vars in 1384 bytes} [Fri Oct 28 09:58:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 28 09:58:51.229249 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:51.252909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a9e9a-14d1-4bee-9073-db478f292d9d None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:51.258438 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:51.259581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:51.260103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:51.260614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:51.264060 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:51.284227 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:51.286299 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:51.325552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1397/2794] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:58:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:58:54.836353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:54.900885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20aba1-fe0c-4409-8ff4-33e5e80e9539 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:54.905820 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:54.906445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:54.906721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:54.907517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] PATH_INFO: `/v3/users/c2771d02bec4465d94e98134015b9705` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:54.908665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2771d02bec4465d94e98134015b9705)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:54.910584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:54.962491 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2771d02bec4465d94e98134015b9705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:54.970420 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2771d02bec4465d94e98134015b9705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:54.982192 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2771d02bec4465d94e98134015b9705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:54.982814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2771d02bec4465d94e98134015b9705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:54.990470 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2771d02bec4465d94e98134015b9705'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:54.996370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1398/2795] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:58:54 2022] DELETE /identity/v3/users/c2771d02bec4465d94e98134015b9705 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:58:55.177836 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:55.188599 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923295b-5028-4756-9ad6-bba7bc856bc8 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:55.246482 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:55.247106 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:55.247384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:55.247650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] PATH_INFO: `/v3/projects/1659fef961ae4ccda1adf6b62c018c9e` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:55.248782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1659fef961ae4ccda1adf6b62c018c9e)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:55.250948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:55.296994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1659fef961ae4ccda1adf6b62c018c9e', 'request_id': 'req-6c6dd747-9a70-4b59-b600-f6e8d470677d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.302587 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1659fef961ae4ccda1adf6b62c018c9e', 'request_id': 'req-6c6dd747-9a70-4b59-b600-f6e8d470677d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.308785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1659fef961ae4ccda1adf6b62c018c9e', 'request_id': 'req-6c6dd747-9a70-4b59-b600-f6e8d470677d'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.321513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] The token cache is being invalidate because project 1659fef961ae4ccda1adf6b62c018c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:58:55.321794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.322410 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.323914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1398/2796] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:58:55 2022] DELETE /identity/v3/projects/1659fef961ae4ccda1adf6b62c018c9e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:58:55.328064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:55.330053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:55.342797 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d087cb5-5ae1-494f-aad6-513fbfb205ae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:55.389801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:58:55.400807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:55.401433 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:55.401704 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:55.401970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] PATH_INFO: `/v3/users/a6a08ed206484313bdab63aa8e6aaa20` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:55.403089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a08ed206484313bdab63aa8e6aaa20)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:55.405053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:55.451062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a08ed206484313bdab63aa8e6aaa20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.459055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a08ed206484313bdab63aa8e6aaa20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.471814 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a08ed206484313bdab63aa8e6aaa20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.472454 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a08ed206484313bdab63aa8e6aaa20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.479440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-db245747-a64e-4cc6-8501-c720cdba0c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a08ed206484313bdab63aa8e6aaa20'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.485253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1399/2797] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:58:55 2022] DELETE /identity/v3/users/a6a08ed206484313bdab63aa8e6aaa20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:58:55.671865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:55.673392 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:55.691767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6dd747-9a70-4b59-b600-f6e8d470677d admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:55.760969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:55.761778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:55.762155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:55.762535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] PATH_INFO: `/v3/projects/80c731d304704ac38498506b9f6a2b20` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:55.763841 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c731d304704ac38498506b9f6a2b20)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:55.765891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:55.778441 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:58:55.816153 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c731d304704ac38498506b9f6a2b20', 'request_id': 'req-52053946-0037-4690-aea5-4f70bc672aa0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.821744 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c731d304704ac38498506b9f6a2b20', 'request_id': 'req-52053946-0037-4690-aea5-4f70bc672aa0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.827792 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c731d304704ac38498506b9f6a2b20', 'request_id': 'req-52053946-0037-4690-aea5-4f70bc672aa0'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.841018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] The token cache is being invalidate because project 80c731d304704ac38498506b9f6a2b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:58:55.841372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.842553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-52053946-0037-4690-aea5-4f70bc672aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:58:55.843377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1399/2798] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:58:55 2022] DELETE /identity/v3/projects/80c731d304704ac38498506b9f6a2b20 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:58:55.849553 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:55.850071 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:55.850182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:55.870331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:58:55.871531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:55.931304 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1400/2799] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:58:55.939180 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:55.939462 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:55.939762 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:55.961040 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:58:55.962277 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:55.970686 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1400/2800] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:58:55.974868 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:55.984367 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef650cf-edcd-430d-bed5-31e6b1cc6fd7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:55.989444 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:55.990038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:55.990311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:55.990580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:55.991215 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:58:55.991515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:55.993159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.005292 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1401/2801] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:58:56.009477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:56.018597 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49ea47-530d-43f7-829a-3f0f75b532f5 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:56.023271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:56.023905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.024183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.024452 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.025076 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:56.026945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.054557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1401/2802] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:58:56 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:58:56.058784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:56.069177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0319b8-47bb-493b-b341-74fb437b302e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:56.128415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:56.129025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.129305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.129577 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.130220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:56.132185 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.140479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:58:56.155116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1402/2803] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:58:56 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:58:56.160094 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:56.170458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c5704-eceb-4449-bed2-5af576963398 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:56.175875 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:56.176474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.176751 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.177020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.177659 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:56.179247 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.189852 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1402/2804] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:58:56.193859 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:56.203610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ab0f5-d5b8-4f2f-86ff-448ed6d4af6f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:56.208432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c8544-2285-4eb5-a3fd-e0365e1c9777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:56.209003 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c8544-2285-4eb5-a3fd-e0365e1c9777 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.209331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c8544-2285-4eb5-a3fd-e0365e1c9777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.209544 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c8544-2285-4eb5-a3fd-e0365e1c9777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.210138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c8544-2285-4eb5-a3fd-e0365e1c9777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:56.211722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c8544-2285-4eb5-a3fd-e0365e1c9777 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.222217 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1403/2805] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:58:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:58:56.226351 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:56.236706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fe474-71f8-4be6-8b3a-7ea17addcc88 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:56.241237 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:56.241878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.242148 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.242415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] PATH_INFO: `/v3/projects/3ce53f0f4dec4042a2cd5a3213b82f5f/users/5b9bb73c501c444ca98c8af9d7b4c7bd/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.258038 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce53f0f4dec4042a2cd5a3213b82f5f, user_id=5b9bb73c501c444ca98c8af9d7b4c7bd, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:56.260698 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.273233 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1403/2806] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:58:56 2022] PUT /identity/v3/projects/3ce53f0f4dec4042a2cd5a3213b82f5f/users/5b9bb73c501c444ca98c8af9d7b4c7bd/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:58:56.278305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2009-daa4-44de-9233-4354a1971d49 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.278586 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2009-daa4-44de-9233-4354a1971d49 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.278853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2009-daa4-44de-9233-4354a1971d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.298621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8a3d2009-daa4-44de-9233-4354a1971d49 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:58:56.299843 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a3d2009-daa4-44de-9233-4354a1971d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:56.357157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1404/2807] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:58:56.367417 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:58:56.369158 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:58:56.425563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b7b50-e294-4a59-9611-2c0cd1672e10 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:58:56.429155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:58:56.429750 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:58:56.430020 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:58:56.430286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:58:56.439670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:58:56.441117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:58:56.450464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1404/2808] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:59:00.218662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.219183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.219311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.248207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] MFA Rules not processed for user `5b9bb73c501c444ca98c8af9d7b4c7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:00.249390 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:00.317882 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1405/2809] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:00.322390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:00.332845 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1fa3b-449a-4fd6-9b65-1ab74f3f8862 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:00.337691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:00.338494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.340312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.340724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.341913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:00.343945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:00.376839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1405/2810] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:00 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:00.378025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:00.388997 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceaaca9-2ba0-486f-8717-6b8701d4d260 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:00.455526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:00.456197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.456523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.456801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.457439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:00.459430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:00.478772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:00.486860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1406/2811] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:00 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 28 09:59:00.490892 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:00.502730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085cf78-6e26-4f17-98f6-4645884705fd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:00.510053 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:00.510933 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.511225 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.511582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.512458 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:00.514740 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:00.527630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1406/2812] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:59:00.532445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:00.542517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b2c74-7172-48f3-bccb-f03da8a6ee4d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:00.548011 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:00.548195 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.548353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.548615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] PATH_INFO: `/v3/projects/fca3f91c0f054e6f93c94479765057e7/users/38b9564e0afa4819bb06ea262da591c6/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.564045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca3f91c0f054e6f93c94479765057e7, user_id=38b9564e0afa4819bb06ea262da591c6, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:00.566759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:00.581212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1407/2813] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:00 2022] PUT /identity/v3/projects/fca3f91c0f054e6f93c94479765057e7/users/38b9564e0afa4819bb06ea262da591c6/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:59:00.585286 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:00.596464 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640edee-fcfb-404a-b0cd-ec8bf69b3762 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:00.600228 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:00.677727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ad9e6-72af-4dc5-8f49-5f60aef8c67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:00.678343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad9e6-72af-4dc5-8f49-5f60aef8c67f admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.678620 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad9e6-72af-4dc5-8f49-5f60aef8c67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.678891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ad9e6-72af-4dc5-8f49-5f60aef8c67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.679531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad9e6-72af-4dc5-8f49-5f60aef8c67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:00.681154 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ad9e6-72af-4dc5-8f49-5f60aef8c67f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:00.697786 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1407/2814] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:59:00.698908 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:00.712097 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f9605-371c-4ef7-a3a3-0572dfbca363 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:00.716905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:00.717578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:00.717852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:00.718126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] PATH_INFO: `/v3/projects/fca3f91c0f054e6f93c94479765057e7/users/38b9564e0afa4819bb06ea262da591c6/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:00.720676 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca3f91c0f054e6f93c94479765057e7, user_id=38b9564e0afa4819bb06ea262da591c6, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:00.723336 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:00.734669 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1408/2815] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:00 2022] PUT /identity/v3/projects/fca3f91c0f054e6f93c94479765057e7/users/38b9564e0afa4819bb06ea262da591c6/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:05.001642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98809776-0bea-4cd8-9a20-28060d1883f5 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:05.002095 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98809776-0bea-4cd8-9a20-28060d1883f5 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:05.002241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98809776-0bea-4cd8-9a20-28060d1883f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:05.030557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-98809776-0bea-4cd8-9a20-28060d1883f5 None None] MFA Rules not processed for user `38b9564e0afa4819bb06ea262da591c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:05.032275 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-98809776-0bea-4cd8-9a20-28060d1883f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:05.130763 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1408/2816] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:05.149883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:05.151987 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:05.239084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9e9b-d94d-4bec-8ff9-80b49caeaa22 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:05.245064 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:05.245700 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:05.245971 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:05.246241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:05.307037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:05.308723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:05.323919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1409/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:06.447469 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:06.448024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:06.448145 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:06.468780 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] MFA Rules not processed for user `5b9bb73c501c444ca98c8af9d7b4c7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:06.470212 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:06.477911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1409/2818] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:59:15.316259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:16.462927 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=63 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:34.725785 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:34.727189 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:34.783822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485880c9-8c25-4b05-a109-046ef66bd3f6 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:34.787410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:34.788062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:34.788335 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:34.788605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:34.797751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:34.799119 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:34.809157 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1410/2819] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:39.686234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:39.696063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be8021-9efe-4358-8fa1-7f6cd8f5cd44 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:39.699517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:39.700147 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:39.700420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:39.700685 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:39.702380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:39.764333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:39.765876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:39.775679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1410/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:42.563954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:42.565589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:42.637911 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:42.639449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:42.652358 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254b7e6a-5573-4fbd-b2cc-2c8402d75cd5 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:42.656946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29137b6b-0647-499d-94a9-58b49c694c1f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:42.657297 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:42.658024 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:42.658372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:42.658706 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] PATH_INFO: `/v3/users/d38ce75423b5412d90c1158396978b97` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:42.660455 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38ce75423b5412d90c1158396978b97)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:42.662386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:42.673729 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:42.674371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:42.675506 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:42.676529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:42.678378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:42.691211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:42.693492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:42.705526 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1411/2821] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 09:59:42.714364 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38ce75423b5412d90c1158396978b97'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:42.722884 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38ce75423b5412d90c1158396978b97'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:42.730465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38ce75423b5412d90c1158396978b97'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:42.731107 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38ce75423b5412d90c1158396978b97'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:42.744436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d56459d0-ab52-4345-bb06-eec4a8526dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38ce75423b5412d90c1158396978b97'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:42.750488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1411/2822] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:59:42 2022] DELETE /identity/v3/users/d38ce75423b5412d90c1158396978b97 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:59:42.928888 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:42.940311 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e764b-c6f7-4ff3-998b-1fff52cb6cf4 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.000937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.001813 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.002087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.002351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] PATH_INFO: `/v3/projects/68e60d7c624e49e1bb43188ae9d7dd2b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.003441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e60d7c624e49e1bb43188ae9d7dd2b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.005399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.041038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:43.051925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e60d7c624e49e1bb43188ae9d7dd2b', 'request_id': 'req-582e4b52-23f4-4bb8-b690-df0f2b9aa351'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:43.060024 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e60d7c624e49e1bb43188ae9d7dd2b', 'request_id': 'req-582e4b52-23f4-4bb8-b690-df0f2b9aa351'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:43.066739 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e60d7c624e49e1bb43188ae9d7dd2b', 'request_id': 'req-582e4b52-23f4-4bb8-b690-df0f2b9aa351'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:43.080110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] The token cache is being invalidate because project 68e60d7c624e49e1bb43188ae9d7dd2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:59:43.080400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:43.081046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-582e4b52-23f4-4bb8-b690-df0f2b9aa351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:43.084435 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1412/2823] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:59:42 2022] DELETE /identity/v3/projects/68e60d7c624e49e1bb43188ae9d7dd2b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:43.088922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.089202 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.089471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.109704 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:43.110820 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:43.173530 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1412/2824] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 28 09:59:43.182623 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.182989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.183356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.203135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:43.204373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:43.213123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1413/2825] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:43.217456 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.228011 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dbd191-6afa-4c62-8363-c4513e3cb407 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.232794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.233460 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.233735 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.234000 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.234667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:59:43.234966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.236603 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.238187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:43.250013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1413/2826] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:59:43.254508 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.263718 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4d58a-1eea-4a8a-9cd6-c3fefd73e764 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.268487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.269075 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.269854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.270120 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.270735 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.272668 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.300067 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1414/2827] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:43.304381 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.314109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8ed01-be28-45e5-9ddd-1c9996a4d159 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.370794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.371350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.371604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.371905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.372543 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.374429 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.396917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1414/2828] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:43 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:43.400340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.411099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d106b-e832-4b96-a290-f4e9ec0c8a3f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.415647 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.416257 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.416529 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.416794 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.417421 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.419001 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.428763 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1415/2829] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 09:59:43.432767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.443285 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04648-f7ed-4fb4-b1af-766e2c9f5c84 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.447832 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6ccc7-1c0d-4873-bfb6-152f1ec6fd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.448426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6ccc7-1c0d-4873-bfb6-152f1ec6fd12 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.448701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6ccc7-1c0d-4873-bfb6-152f1ec6fd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.448968 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6ccc7-1c0d-4873-bfb6-152f1ec6fd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.449607 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6ccc7-1c0d-4873-bfb6-152f1ec6fd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.451183 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6ccc7-1c0d-4873-bfb6-152f1ec6fd12 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.463774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1415/2830] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:43.468181 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.478175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83544d71-1076-44fb-a72e-ba4bb623fae1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.482783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.483411 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.483684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.483980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] PATH_INFO: `/v3/projects/75f124edebb64b4191ec48450e1c272d/users/bede338290734621b054afe6eba59ad3/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.498002 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f124edebb64b4191ec48450e1c272d, user_id=bede338290734621b054afe6eba59ad3, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.500671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.515122 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1416/2831] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:43 2022] PUT /identity/v3/projects/75f124edebb64b4191ec48450e1c272d/users/bede338290734621b054afe6eba59ad3/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:59:43.525891 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.526197 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.526471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.553976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] MFA Rules not processed for user `bede338290734621b054afe6eba59ad3`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:43.555193 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:43.619950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1416/2832] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:43.631864 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:43.633487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:43.707396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06116538-c5c8-45fb-a714-ba65c2b04a54 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:43.713228 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a26e51-c5f8-46c3-bdf0-81c3ec21c07b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:43.713833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26e51-c5f8-46c3-bdf0-81c3ec21c07b service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:43.714111 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26e51-c5f8-46c3-bdf0-81c3ec21c07b service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:43.714384 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a26e51-c5f8-46c3-bdf0-81c3ec21c07b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:43.722186 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a26e51-c5f8-46c3-bdf0-81c3ec21c07b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:43.723934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a26e51-c5f8-46c3-bdf0-81c3ec21c07b service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:43.732640 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1417/2833] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:48.927048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:48.990991 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c31897-a281-4eb7-9331-d6863d950c0a None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:48.995616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:48.996282 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:48.996557 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:48.996825 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] PATH_INFO: `/v3/users/bede338290734621b054afe6eba59ad3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:48.997941 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bede338290734621b054afe6eba59ad3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.000018 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.002996 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:49.048139 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bede338290734621b054afe6eba59ad3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.055917 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bede338290734621b054afe6eba59ad3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.063236 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bede338290734621b054afe6eba59ad3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.063939 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bede338290734621b054afe6eba59ad3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.078535 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bede338290734621b054afe6eba59ad3'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.084715 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1417/2834] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:59:48 2022] DELETE /identity/v3/users/bede338290734621b054afe6eba59ad3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:49.089149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.089636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.089753 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.108202 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:49.109401 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:49.149851 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:49.171224 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1418/2835] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:49.181946 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:49.183449 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:49.248418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2888c66-793f-41a9-9d1c-4650d605b2a3 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:49.252443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de39f5-a1e9-46a7-ba2a-ec088df2392b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:49.253135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de39f5-a1e9-46a7-ba2a-ec088df2392b service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.253418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de39f5-a1e9-46a7-ba2a-ec088df2392b service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.253688 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de39f5-a1e9-46a7-ba2a-ec088df2392b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.264073 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de39f5-a1e9-46a7-ba2a-ec088df2392b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.265556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de39f5-a1e9-46a7-ba2a-ec088df2392b service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.275774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1418/2836] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:49.547986 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:49.549410 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:49.563461 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adc39d-32b6-4a1d-923b-5a99cbed5b54 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:49.568938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:49.569662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.569942 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.570212 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] PATH_INFO: `/v3/projects/75f124edebb64b4191ec48450e1c272d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.571363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f124edebb64b4191ec48450e1c272d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.573428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.622264 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f124edebb64b4191ec48450e1c272d', 'request_id': 'req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.629680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f124edebb64b4191ec48450e1c272d', 'request_id': 'req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.635345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f124edebb64b4191ec48450e1c272d', 'request_id': 'req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.646331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] The token cache is being invalidate because project 75f124edebb64b4191ec48450e1c272d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:59:49.646615 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.647269 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa747289-1cf6-4efc-8d5b-8d49513ecfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:49.648773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1419/2837] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:59:49 2022] DELETE /identity/v3/projects/75f124edebb64b4191ec48450e1c272d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:49.653584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.653866 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.654151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.674486 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:49.675679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:49.733923 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1419/2838] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:49.742666 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.742953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.743225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.762695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:49.763943 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:49.772347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1420/2839] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:49.776494 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:49.786118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6259fb-a9eb-4e11-8480-b307bec03a61 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:49.791653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:49.792314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.792596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.792864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.793515 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:59:49.793818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.795415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.807465 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1420/2840] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:59:49.811881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:49.821063 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2614e4-7c4f-4a8b-a410-0a9fbb02fb5c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:49.825681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:49.826270 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.826548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.826822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.827457 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.829397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.858307 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1421/2841] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:49 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:49.862474 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:49.872797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1f62c-bc39-4231-9d5f-e1fee940206e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:49.881136 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:49.931732 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:49.932767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.932767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.933051 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.933512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.935411 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.958089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1421/2842] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:49 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:49.962701 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:49.973164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3293fc9-4d93-4283-b1f6-de0df124b6ef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:49.978460 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:49.979049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:49.979325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:49.979594 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:49.980268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:49.981862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:49.991714 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1422/2843] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:49.995656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.006013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3592d-8a92-4f1d-b153-e120c5bac6bd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.010630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c288baf-ec74-4514-949b-afd54fb57a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.011216 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288baf-ec74-4514-949b-afd54fb57a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.011661 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288baf-ec74-4514-949b-afd54fb57a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.011993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c288baf-ec74-4514-949b-afd54fb57a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.012683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c288baf-ec74-4514-949b-afd54fb57a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.014291 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c288baf-ec74-4514-949b-afd54fb57a09 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.025495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1422/2844] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:50.029528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.040016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3735fae-914f-4957-a961-5043402d897f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.044621 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.045258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.045645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.045925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] PATH_INFO: `/v3/projects/c7ab0eba53b24b71afccd4103e98ed92/users/ee5a99d174df4d19a2fab033513cfa6a/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.060297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ab0eba53b24b71afccd4103e98ed92, user_id=ee5a99d174df4d19a2fab033513cfa6a, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.062961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.078278 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1423/2845] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:50 2022] PUT /identity/v3/projects/c7ab0eba53b24b71afccd4103e98ed92/users/ee5a99d174df4d19a2fab033513cfa6a/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:50.090310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.090590 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.090855 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.118958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] MFA Rules not processed for user `ee5a99d174df4d19a2fab033513cfa6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:50.120205 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:50.191379 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1423/2846] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:50.196512 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.256983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8acf3-b52c-42b9-8c6f-159f87ffc803 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.261620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.262221 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.262500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.262773 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.263399 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.265422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.275239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:50.294240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1424/2847] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:50 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:50.298354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.309296 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b123c5-f5bd-411f-8bb5-8fbd67528760 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.364954 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.365562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.365839 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.366108 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.366746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.368694 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.389961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1424/2848] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:50 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:50.394168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.405110 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0c539-049d-4aef-a2da-fca166189175 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.409655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.410252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.410525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.410792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.411450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.413092 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.423050 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1425/2849] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:50.427175 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.437271 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fcaea3-77e7-47ee-9b59-70bee57f0f48 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.441976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.442656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.443443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.443712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] PATH_INFO: `/v3/projects/e24f501338854d62900696a344a6f479/users/64c15ae9a11249f3bf7e23c38fd66e46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.446720 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:50.459687 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24f501338854d62900696a344a6f479, user_id=64c15ae9a11249f3bf7e23c38fd66e46, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.462407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.474926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1425/2850] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:50 2022] PUT /identity/v3/projects/e24f501338854d62900696a344a6f479/users/64c15ae9a11249f3bf7e23c38fd66e46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:50.478937 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.488923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8804e-1bed-48b8-99bf-813cf4a798c4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.545573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad49252-b725-49d4-b98f-226e4a496263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.546177 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad49252-b725-49d4-b98f-226e4a496263 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.546456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad49252-b725-49d4-b98f-226e4a496263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.546723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad49252-b725-49d4-b98f-226e4a496263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.547347 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad49252-b725-49d4-b98f-226e4a496263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.548988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad49252-b725-49d4-b98f-226e4a496263 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.558976 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1426/2851] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:50.563273 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.573990 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad5458-d68c-4069-8f34-85aab025b302 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.578714 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.579552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.579853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.580130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] PATH_INFO: `/v3/projects/e24f501338854d62900696a344a6f479/users/64c15ae9a11249f3bf7e23c38fd66e46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.582113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24f501338854d62900696a344a6f479, user_id=64c15ae9a11249f3bf7e23c38fd66e46, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.584767 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.595186 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1426/2852] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:50 2022] PUT /identity/v3/projects/e24f501338854d62900696a344a6f479/users/64c15ae9a11249f3bf7e23c38fd66e46/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:50.656806 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.657033 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.657268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.686038 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] MFA Rules not processed for user `64c15ae9a11249f3bf7e23c38fd66e46`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:50.687252 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:50.755077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1427/2853] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:50.766241 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:50.767949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:50.825314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5326c-b94e-45bf-bda5-14226f693731 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:50.828950 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85f1a0e-c3e0-4b56-b1ee-d1b55f429f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:50.829545 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f1a0e-c3e0-4b56-b1ee-d1b55f429f44 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:50.829818 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f1a0e-c3e0-4b56-b1ee-d1b55f429f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:50.830089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85f1a0e-c3e0-4b56-b1ee-d1b55f429f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:50.885219 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f1a0e-c3e0-4b56-b1ee-d1b55f429f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:50.886683 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85f1a0e-c3e0-4b56-b1ee-d1b55f429f44 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:50.895722 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1427/2854] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:53.436754 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:53.497938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082ca85-27f9-4d38-b2fd-7de16ef98f71 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:53.502630 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:53.503253 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:53.503527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:53.503833 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] PATH_INFO: `/v3/users/ee5a99d174df4d19a2fab033513cfa6a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:53.504954 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee5a99d174df4d19a2fab033513cfa6a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:53.506871 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:53.508964 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:53.554200 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee5a99d174df4d19a2fab033513cfa6a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:53.561477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee5a99d174df4d19a2fab033513cfa6a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:53.574090 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee5a99d174df4d19a2fab033513cfa6a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:53.574774 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee5a99d174df4d19a2fab033513cfa6a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:53.581492 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee5a99d174df4d19a2fab033513cfa6a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:53.587885 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1428/2855] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:59:53 2022] DELETE /identity/v3/users/ee5a99d174df4d19a2fab033513cfa6a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 09:59:53.592380 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:53.592872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:53.592972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:53.611693 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:53.612916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:53.671529 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1428/2856] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:53.681256 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:53.682660 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:53.737915 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25328e02-b599-420a-9fe0-0006fc1a07d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:53.741521 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:53.742170 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:53.742440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:53.742703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:53.751424 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:53.753159 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:53.762916 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1429/2857] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:54.029321 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:54.030953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:54.043215 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad71b18-163b-4537-af93-d9879df5d921 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:54.048142 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:54.048853 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.049194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.049539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] PATH_INFO: `/v3/projects/c7ab0eba53b24b71afccd4103e98ed92` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.050840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ab0eba53b24b71afccd4103e98ed92)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:54.052840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:54.079375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:54.103507 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ab0eba53b24b71afccd4103e98ed92', 'request_id': 'req-8d8ada51-9da5-426a-aa41-e0392e786032'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.108949 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ab0eba53b24b71afccd4103e98ed92', 'request_id': 'req-8d8ada51-9da5-426a-aa41-e0392e786032'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.115144 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ab0eba53b24b71afccd4103e98ed92', 'request_id': 'req-8d8ada51-9da5-426a-aa41-e0392e786032'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.128581 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] The token cache is being invalidate because project c7ab0eba53b24b71afccd4103e98ed92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:59:54.128872 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.129514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.130993 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1429/2858] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:59:54 2022] DELETE /identity/v3/projects/c7ab0eba53b24b71afccd4103e98ed92 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:54.135950 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:54.145731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ce244-7fb0-46d6-b5eb-0c74dbced061 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:54.202275 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:54.202897 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.203168 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.203432 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] PATH_INFO: `/v3/users/64c15ae9a11249f3bf7e23c38fd66e46` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.204575 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c15ae9a11249f3bf7e23c38fd66e46)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:54.206487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:54.230053 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:54.254357 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c15ae9a11249f3bf7e23c38fd66e46'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.262343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c15ae9a11249f3bf7e23c38fd66e46'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.274905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c15ae9a11249f3bf7e23c38fd66e46'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.275536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c15ae9a11249f3bf7e23c38fd66e46'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.282324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-aa48dd34-b3e4-4e72-89e8-48320ea2ed1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c15ae9a11249f3bf7e23c38fd66e46'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.288417 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1430/2859] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 09:59:54 2022] DELETE /identity/v3/users/64c15ae9a11249f3bf7e23c38fd66e46 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:54.551254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:54.552717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:54.567316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ada51-9da5-426a-aa41-e0392e786032 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:54.631479 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:54.632343 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.633251 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.633584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] PATH_INFO: `/v3/projects/e24f501338854d62900696a344a6f479` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.634812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24f501338854d62900696a344a6f479)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:54.643967 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:54.694363 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24f501338854d62900696a344a6f479', 'request_id': 'req-bf1563c3-4680-495a-a43f-bb8f99378f91'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.701443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24f501338854d62900696a344a6f479', 'request_id': 'req-bf1563c3-4680-495a-a43f-bb8f99378f91'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.708372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24f501338854d62900696a344a6f479', 'request_id': 'req-bf1563c3-4680-495a-a43f-bb8f99378f91'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.720893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] The token cache is being invalidate because project e24f501338854d62900696a344a6f479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 09:59:54.721305 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.721931 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-bf1563c3-4680-495a-a43f-bb8f99378f91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 09:59:54.723699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1430/2860] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 09:59:54 2022] DELETE /identity/v3/projects/e24f501338854d62900696a344a6f479 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:54.728434 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.728905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.729030 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.749165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:54.750344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:54.815214 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1431/2861] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:54.823552 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.823865 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.824143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.844042 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:54.845232 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:54.855547 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1431/2862] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:54.859852 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:54.870116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5499d2-b880-4fde-9009-b02de54095c4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:54.874784 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:54.875423 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.875665 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.875970 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.876613 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 09:59:54.877167 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:54.878768 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:54.890930 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1432/2863] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 09:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 09:59:54.895257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:54.904914 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acb464-6bda-487a-aa6f-3bc3b5d41383 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:54.909488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:54.910126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:54.910407 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:54.910670 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:54.911293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:54.913174 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:54.939790 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1432/2864] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:54 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:54.943988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:54.954266 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492e17c-9534-466c-9f00-49065568088e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.011025 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.011545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.011853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.012130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.012761 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.014678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.036798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1433/2865] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:54 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:55.041386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.051662 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4269a64-9cea-4f27-ad67-cc28e20e54dd admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.056329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.056920 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.057199 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.057471 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.058112 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.059701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.065160 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:55.072442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1433/2866] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:55.075240 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.086341 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8229cd3-9376-4762-825f-96596fdfbcc7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.091009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e91834a-c694-496b-acf8-df62a35a31b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.091636 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91834a-c694-496b-acf8-df62a35a31b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.091947 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91834a-c694-496b-acf8-df62a35a31b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.092232 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e91834a-c694-496b-acf8-df62a35a31b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.092854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91834a-c694-496b-acf8-df62a35a31b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.094487 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e91834a-c694-496b-acf8-df62a35a31b8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.105182 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1434/2867] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:55.109365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.119384 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac49bc5-e0db-4147-ad28-fe44d391f426 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.124013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.124654 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.124928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.125198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] PATH_INFO: `/v3/projects/f2952fd10acb4000acfcdab43bcb4927/users/a2c0926bdee947a2acb010c460b09595/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.141235 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2952fd10acb4000acfcdab43bcb4927, user_id=a2c0926bdee947a2acb010c460b09595, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.143904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.157976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1434/2868] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:55 2022] PUT /identity/v3/projects/f2952fd10acb4000acfcdab43bcb4927/users/a2c0926bdee947a2acb010c460b09595/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:55.169527 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205b1e-6061-417d-9776-251769aeef77 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.169815 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205b1e-6061-417d-9776-251769aeef77 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.170095 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78205b1e-6061-417d-9776-251769aeef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.183580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:55.199197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-78205b1e-6061-417d-9776-251769aeef77 None None] MFA Rules not processed for user `a2c0926bdee947a2acb010c460b09595`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:55.200430 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-78205b1e-6061-417d-9776-251769aeef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:55.263872 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1435/2869] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:55.268238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.330017 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1622-c619-4c2e-a4f6-279e205ce18a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.334653 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.335254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.335533 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.336387 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.337013 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.338902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.367307 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1435/2870] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:55 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:55.369324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78205b1e-6061-417d-9776-251769aeef77 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.380277 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78205b1e-6061-417d-9776-251769aeef77 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.442522 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.443130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.443414 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.443733 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.444344 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.446807 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.467854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1436/2871] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:55 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:55.471953 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.482797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a9835-0cc3-4267-a597-a1fd87c8130b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.487325 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.487948 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.488221 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.488488 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.489116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.490713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.502973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1436/2872] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:55.506884 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.517542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc720b9f-8324-41e6-ab6b-5ab07cb7c60c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.522052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c2297-2dde-4abc-b57c-eaf28a01343b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.522645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2297-2dde-4abc-b57c-eaf28a01343b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.522919 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2297-2dde-4abc-b57c-eaf28a01343b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.523187 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2297-2dde-4abc-b57c-eaf28a01343b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.523862 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c2297-2dde-4abc-b57c-eaf28a01343b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.525456 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c2297-2dde-4abc-b57c-eaf28a01343b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.535291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1437/2873] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:55.539470 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.549375 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543a656-5181-48d2-afb5-e3291dfaf313 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.553945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.555117 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.555382 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.555646 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] PATH_INFO: `/v3/projects/f149f59ca9e24771ab10e05f494b89d7/users/00b17c41207f4a7cb0e16e276804f921/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.570726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f149f59ca9e24771ab10e05f494b89d7, user_id=00b17c41207f4a7cb0e16e276804f921, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.573513 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.586426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1437/2874] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:55 2022] PUT /identity/v3/projects/f149f59ca9e24771ab10e05f494b89d7/users/00b17c41207f4a7cb0e16e276804f921/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:55.596854 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.597165 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.597439 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.625418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] MFA Rules not processed for user `00b17c41207f4a7cb0e16e276804f921`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:55.626648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:55.690291 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1438/2875] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:55.694676 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.755553 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b18d-0048-4b5e-b9b0-bea33c163d1a admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.760185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.760781 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.761057 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.761323 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.761940 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.763849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.791334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1438/2876] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 09:59:55 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:55.794597 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.801219 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:55.806089 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f961fe-113e-4dd5-b5a6-129e425d981b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.861409 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.862017 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.862293 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.862562 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.863188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.865751 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.886363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1439/2877] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 09:59:55 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 09:59:55.890372 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.900556 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4156a2-84a5-482d-9f51-074d4ec0992b admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.905036 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.905623 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.905893 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.906159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.906785 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.908390 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.913427 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:55.920234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1439/2878] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:55.924045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.935558 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ac073-55ee-4493-855d-859ec85c6955 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:55.940234 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:55.940881 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:55.941158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:55.941429 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] PATH_INFO: `/v3/projects/c4e1556b77854d338c00797b328f731e/users/51da7b3ba5e74c19bfaed7f53c206f56/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:55.956440 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e1556b77854d338c00797b328f731e, user_id=51da7b3ba5e74c19bfaed7f53c206f56, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:55.959091 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:55.971297 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1440/2879] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:55 2022] PUT /identity/v3/projects/c4e1556b77854d338c00797b328f731e/users/51da7b3ba5e74c19bfaed7f53c206f56/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:55.975453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:55.985974 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ca020-e8a0-403d-9301-e99fb5a20863 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:56.041021 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a7cae-2649-4eaa-9096-ce55896f17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:56.041671 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7cae-2649-4eaa-9096-ce55896f17a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:56.041900 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7cae-2649-4eaa-9096-ce55896f17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:56.042172 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a7cae-2649-4eaa-9096-ce55896f17a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:56.042803 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a7cae-2649-4eaa-9096-ce55896f17a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:56.044453 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a7cae-2649-4eaa-9096-ce55896f17a9 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:56.053997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1440/2880] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 09:59:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 09:59:56.058695 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:56.069070 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ce9a9-24ce-4662-a933-7daefee5a066 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:56.073775 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:56.074428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:56.074703 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:56.074973 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] PATH_INFO: `/v3/projects/c4e1556b77854d338c00797b328f731e/users/51da7b3ba5e74c19bfaed7f53c206f56/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:56.076911 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e1556b77854d338c00797b328f731e, user_id=51da7b3ba5e74c19bfaed7f53c206f56, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:56.079554 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:56.089940 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1441/2881] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 09:59:56 2022] PUT /identity/v3/projects/c4e1556b77854d338c00797b328f731e/users/51da7b3ba5e74c19bfaed7f53c206f56/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 09:59:56.100334 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:56.100731 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:56.101726 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:56.131778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] MFA Rules not processed for user `51da7b3ba5e74c19bfaed7f53c206f56`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 09:59:56.133116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:56.202110 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1441/2882] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 09:59:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 09:59:56.214040 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:56.215798 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:56.272671 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324af2b-e6cb-4c73-94b1-429949728d2c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:56.276867 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:56.277467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:56.277750 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:56.278045 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:56.332193 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:56.333567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:56.342795 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1442/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:57.844971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:57.846405 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 09:59:57.858776 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05619dcf-6543-487c-b98a-efedd3e425df None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:57.862306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:57.862902 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:57.863177 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:57.863447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:57.919558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:57.920965 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:57.929798 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1442/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 09:59:58.474845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 09:59:58.483510 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1c534-ab8f-4279-9e8c-b39b540a1f13 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 09:59:58.487220 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249980b-a013-4752-bc82-4b0ccfe6055d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 09:59:58.487847 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249980b-a013-4752-bc82-4b0ccfe6055d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 09:59:58.488123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249980b-a013-4752-bc82-4b0ccfe6055d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 09:59:58.488394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249980b-a013-4752-bc82-4b0ccfe6055d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 09:59:58.497845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249980b-a013-4752-bc82-4b0ccfe6055d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 09:59:58.499276 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249980b-a013-4752-bc82-4b0ccfe6055d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 09:59:58.503345 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 09:59:58.511380 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1443/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 09:59:58 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:02.396432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:02.418157 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=100 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:02.459176 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23e0f6-c07f-4ec2-953f-ddda1d9d269c service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:02.464254 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:02.464880 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:02.465159 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:02.465431 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] PATH_INFO: `/v3/users/a2c0926bdee947a2acb010c460b09595` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:02.466539 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c0926bdee947a2acb010c460b09595)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:02.468489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:02.514958 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c0926bdee947a2acb010c460b09595'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:02.522194 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c0926bdee947a2acb010c460b09595'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:02.529546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c0926bdee947a2acb010c460b09595'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:02.530164 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c0926bdee947a2acb010c460b09595'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:02.542135 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c0926bdee947a2acb010c460b09595'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:02.547822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1443/2886] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:02 2022] DELETE /identity/v3/users/a2c0926bdee947a2acb010c460b09595 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:02.552394 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:02.552879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:02.552989 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:02.571865 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:02.573037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:02.631207 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1444/2887] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:02.643118 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:02.645400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:02.709369 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36e28-9dcc-4bad-805b-7e1cd86ff38e admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:02.713614 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:02.714222 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:02.714495 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:02.714760 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:02.723549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:02.724999 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:02.734302 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1444/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:02.997996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:02.999570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:03.014637 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cd8d8-a677-4ed6-92fe-eb59c58ea396 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:03.019400 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:03.020062 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:03.020340 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:03.020610 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] PATH_INFO: `/v3/projects/f2952fd10acb4000acfcdab43bcb4927` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:03.021720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2952fd10acb4000acfcdab43bcb4927)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:03.023841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:03.069103 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2952fd10acb4000acfcdab43bcb4927', 'request_id': 'req-e80108cb-4d54-4558-b44f-a96c0022ea52'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.076582 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2952fd10acb4000acfcdab43bcb4927', 'request_id': 'req-e80108cb-4d54-4558-b44f-a96c0022ea52'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.083500 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2952fd10acb4000acfcdab43bcb4927', 'request_id': 'req-e80108cb-4d54-4558-b44f-a96c0022ea52'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.095153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] The token cache is being invalidate because project f2952fd10acb4000acfcdab43bcb4927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:03.095445 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.096112 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.097585 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1445/2889] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:02 2022] DELETE /identity/v3/projects/f2952fd10acb4000acfcdab43bcb4927 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:03.101624 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:03.111333 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c39b-668d-479f-a9f9-95b9016cfa66 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:03.152200 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:03.170077 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:03.170650 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:03.170973 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:03.171224 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] PATH_INFO: `/v3/users/00b17c41207f4a7cb0e16e276804f921` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:03.172357 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b17c41207f4a7cb0e16e276804f921)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:03.174259 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:03.219413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b17c41207f4a7cb0e16e276804f921'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.226797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b17c41207f4a7cb0e16e276804f921'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.233787 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b17c41207f4a7cb0e16e276804f921'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.234413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b17c41207f4a7cb0e16e276804f921'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.245782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b17c41207f4a7cb0e16e276804f921'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.251717 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1445/2890] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:03 2022] DELETE /identity/v3/users/00b17c41207f4a7cb0e16e276804f921 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:03.521324 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:03.522736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:03.536310 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80108cb-4d54-4558-b44f-a96c0022ea52 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:03.590720 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:03.591351 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:03.591605 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:03.591905 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] PATH_INFO: `/v3/projects/f149f59ca9e24771ab10e05f494b89d7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:03.592991 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f149f59ca9e24771ab10e05f494b89d7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:03.594896 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:03.619188 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:03.641254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f149f59ca9e24771ab10e05f494b89d7', 'request_id': 'req-cf2f9d6c-e7bd-413a-abe1-83835d0da891'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.649386 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f149f59ca9e24771ab10e05f494b89d7', 'request_id': 'req-cf2f9d6c-e7bd-413a-abe1-83835d0da891'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.654772 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f149f59ca9e24771ab10e05f494b89d7', 'request_id': 'req-cf2f9d6c-e7bd-413a-abe1-83835d0da891'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.665845 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] The token cache is being invalidate because project f149f59ca9e24771ab10e05f494b89d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:03.666127 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.666767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.668279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1446/2891] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:03 2022] DELETE /identity/v3/projects/f149f59ca9e24771ab10e05f494b89d7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:03.672444 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:03.673868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:03.686856 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42bae6-29d0-4a42-9164-530f0b8a6818 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:03.741353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:03.742442 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:03.742721 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:03.742992 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] PATH_INFO: `/v3/users/51da7b3ba5e74c19bfaed7f53c206f56` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:03.744140 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51da7b3ba5e74c19bfaed7f53c206f56)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:03.746048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:03.790937 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51da7b3ba5e74c19bfaed7f53c206f56'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.797985 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51da7b3ba5e74c19bfaed7f53c206f56'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.805656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51da7b3ba5e74c19bfaed7f53c206f56'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.806279 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51da7b3ba5e74c19bfaed7f53c206f56'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.807359 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:03.820679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-3883492a-98e3-43b2-82c9-0ee6f856110f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51da7b3ba5e74c19bfaed7f53c206f56'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:03.826472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1446/2892] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:03 2022] DELETE /identity/v3/users/51da7b3ba5e74c19bfaed7f53c206f56 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:04.096274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.097755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:04.116039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2f9d6c-e7bd-413a-abe1-83835d0da891 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.172274 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.172902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.173174 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.173441 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] PATH_INFO: `/v3/projects/c4e1556b77854d338c00797b328f731e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.174539 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e1556b77854d338c00797b328f731e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.176471 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.220265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e1556b77854d338c00797b328f731e', 'request_id': 'req-223b21a0-7388-410f-9658-6f4e7283104f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:04.227607 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e1556b77854d338c00797b328f731e', 'request_id': 'req-223b21a0-7388-410f-9658-6f4e7283104f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:04.233178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e1556b77854d338c00797b328f731e', 'request_id': 'req-223b21a0-7388-410f-9658-6f4e7283104f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:04.244653 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] The token cache is being invalidate because project c4e1556b77854d338c00797b328f731e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:04.244933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:04.245548 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-223b21a0-7388-410f-9658-6f4e7283104f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:04.246994 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1447/2893] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:04 2022] DELETE /identity/v3/projects/c4e1556b77854d338c00797b328f731e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:04.251840 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.252391 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.252664 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.272234 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:04.273477 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:04.331074 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1447/2894] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:04.338783 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.339065 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.339338 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.358923 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:04.360150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:04.368759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1448/2895] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:04.372876 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.382684 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5999d5c-20f8-40a6-90ed-73f7c8772bcb None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.387126 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.387718 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.388048 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.388312 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.388945 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81610) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 10:00:04.389240 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.390804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.402725 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1448/2896] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 10:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 10:00:04.406841 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.416242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d44151-83e3-400e-8b02-2e7ae133beac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.420721 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.421301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.421570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.421834 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.422455 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.424378 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.450570 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1449/2897] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:04.454066 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.464527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f576f-4702-4e2b-b6b0-f184436a5c55 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.521171 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.521782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.522062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.522385 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.523006 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.524961 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.545314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1449/2898] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:04.549438 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.559953 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c06c-adbc-4bbd-9191-3ecc7deffe9c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.564397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.564985 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.565258 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.565523 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.566153 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.567792 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.577879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1450/2899] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:04.581656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.591185 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbad497-b257-42c8-8a19-2be5ae4479c2 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.595604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676b546-c3af-40f3-8c13-d931f1fef629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.596229 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676b546-c3af-40f3-8c13-d931f1fef629 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.596499 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676b546-c3af-40f3-8c13-d931f1fef629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.596765 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676b546-c3af-40f3-8c13-d931f1fef629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.597394 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676b546-c3af-40f3-8c13-d931f1fef629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.598951 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676b546-c3af-40f3-8c13-d931f1fef629 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.609143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1450/2900] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 28 10:00:04.613099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.618748 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:04.625104 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bc8c1-4a30-4b4b-b0c4-abbf41d19f7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.631009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.631644 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.631945 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.632216 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] PATH_INFO: `/v3/projects/ad9f9000b995454784def4f7b8c79ee7/users/e252110e5f9448ce8b9617fd17d4fb5b/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.647726 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9f9000b995454784def4f7b8c79ee7, user_id=e252110e5f9448ce8b9617fd17d4fb5b, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.650486 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.663101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1451/2901] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 10:00:04 2022] PUT /identity/v3/projects/ad9f9000b995454784def4f7b8c79ee7/users/e252110e5f9448ce8b9617fd17d4fb5b/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:04.673457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.673928 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.674025 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.702886 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] MFA Rules not processed for user `e252110e5f9448ce8b9617fd17d4fb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:04.704130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:04.736549 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:04.769087 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1451/2902] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:04.773629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.834996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bf986-5423-47fd-8175-de77e4908410 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.839561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.840183 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.840458 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.840724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.841343 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.843227 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.868993 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1452/2903] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:04.873041 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.883724 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a8d6a-7651-471e-b4f5-0d05df6e369c None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.939138 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.939753 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.940043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.940316 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.940952 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.942848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.966265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1452/2904] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 10:00:04 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:04.967468 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:04.978125 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ad76a-0f73-46b6-929f-af79a4448b41 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:04.982699 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:04.983286 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:04.983564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:04.983938 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:04.984578 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:04.986171 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:04.997265 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1453/2905] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:05.001878 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.013582 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb81c9-8c74-43d5-be60-a861e0fe6fcc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.018196 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910460c6-9452-4f99-a9ec-878b16d8bd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.018788 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910460c6-9452-4f99-a9ec-878b16d8bd45 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.019056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910460c6-9452-4f99-a9ec-878b16d8bd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.019317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910460c6-9452-4f99-a9ec-878b16d8bd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.019971 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910460c6-9452-4f99-a9ec-878b16d8bd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.021531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910460c6-9452-4f99-a9ec-878b16d8bd45 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.031493 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1453/2906] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:05.035561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.046123 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81f250-9a50-43f9-9441-c788f4848b5a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.050740 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.051385 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.051678 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.051983 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] PATH_INFO: `/v3/projects/0c9a542ee6cb4f018398266b178ab97b/users/19d01393202c42d8abe87d46267ebb8e/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.067031 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9a542ee6cb4f018398266b178ab97b, user_id=19d01393202c42d8abe87d46267ebb8e, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.069770 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.083225 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1454/2907] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 10:00:05 2022] PUT /identity/v3/projects/0c9a542ee6cb4f018398266b178ab97b/users/19d01393202c42d8abe87d46267ebb8e/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:05.094558 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.094838 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.095123 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.123766 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] MFA Rules not processed for user `19d01393202c42d8abe87d46267ebb8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:05.124975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:05.190604 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1454/2908] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:05.195255 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.255879 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233e1ed-6f6a-405d-a1e5-0a4f5e914b7c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.260528 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.261130 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.261406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.261681 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.262306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.264259 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.292968 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1455/2909] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 10:00:05 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:05.296352 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.306679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be26d3-d2da-497c-8b7c-00951126733e None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.348116 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:05.363970 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.364571 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.364844 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.365109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] PATH_INFO: `/v3/users` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.365728 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.367608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.388713 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1455/2910] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 10:00:05 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:05.392755 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.403525 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7ba6-9f9e-43d1-81c3-d79b3569583a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.408076 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.408662 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.408931 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.409196 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.409831 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.411448 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.422302 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1456/2911] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:05.426701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.439888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82af57-3b74-4f02-87e7-421ec7654af4 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.444969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.445702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.445976 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.446242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] PATH_INFO: `/v3/projects/18ec64147eae4166bed67abe610885c2/users/9639b8124c2d4282ad3a5a77ad6984c9/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.464988 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ec64147eae4166bed67abe610885c2, user_id=9639b8124c2d4282ad3a5a77ad6984c9, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.467629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.481806 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1456/2912] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 10:00:05 2022] PUT /identity/v3/projects/18ec64147eae4166bed67abe610885c2/users/9639b8124c2d4282ad3a5a77ad6984c9/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:05.486801 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.493242 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:05.499980 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2448872-c02d-44f6-9a75-d3990134d567 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.557016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6242479-e2cb-4a95-a03a-a3df28a21118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.557540 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242479-e2cb-4a95-a03a-a3df28a21118 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.557817 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242479-e2cb-4a95-a03a-a3df28a21118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.558087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6242479-e2cb-4a95-a03a-a3df28a21118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.558722 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6242479-e2cb-4a95-a03a-a3df28a21118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.560361 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6242479-e2cb-4a95-a03a-a3df28a21118 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.570479 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1457/2913] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:05.574472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.584583 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b388d-2f18-4cf6-9d43-0de31de3af46 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.589774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.590420 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.590778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.591049 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] PATH_INFO: `/v3/projects/18ec64147eae4166bed67abe610885c2/users/9639b8124c2d4282ad3a5a77ad6984c9/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.592997 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ec64147eae4166bed67abe610885c2, user_id=9639b8124c2d4282ad3a5a77ad6984c9, role_id=9bc312f0ac424947bdcf6e9ed4d9a6f3)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.595652 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.605679 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1457/2914] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 10:00:05 2022] PUT /identity/v3/projects/18ec64147eae4166bed67abe610885c2/users/9639b8124c2d4282ad3a5a77ad6984c9/roles/9bc312f0ac424947bdcf6e9ed4d9a6f3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:05.615531 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.616037 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.616135 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.644306 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] MFA Rules not processed for user `9639b8124c2d4282ad3a5a77ad6984c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:05.645505 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:05.711084 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1458/2915] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:05.722546 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:05.724310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:05.781242 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41214d58-aba9-495f-a0cb-9e12b26a4bbb admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:05.784904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:05.785484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:05.785782 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:05.786043 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:05.840774 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:05.842113 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:05.850857 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1458/2916] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:07.364569 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:07.365978 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:07.378077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551e83-923a-45ab-91dc-9068182c59ca None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:07.381627 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:07.382218 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:07.382496 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:07.382765 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:07.437244 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:07.438679 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:07.447419 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1459/2917] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:08.139579 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:08.148178 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c48231-bd96-4a53-980b-f5df7208fb1b service cinder] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:08.152317 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8529cd-8eb1-40dc-99ae-639f077dba2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:08.152913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8529cd-8eb1-40dc-99ae-639f077dba2f service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:08.153188 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8529cd-8eb1-40dc-99ae-639f077dba2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:08.153484 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8529cd-8eb1-40dc-99ae-639f077dba2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:08.162426 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8529cd-8eb1-40dc-99ae-639f077dba2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:08.163849 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8529cd-8eb1-40dc-99ae-639f077dba2f service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:08.173656 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1459/2918] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:12.030150 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:12.054415 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:12.092588 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5eb7d-fc75-4897-97f4-31a68b1e7bad service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:12.097066 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:12.097684 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:12.097963 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:12.098236 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] PATH_INFO: `/v3/users/e252110e5f9448ce8b9617fd17d4fb5b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:12.099334 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e252110e5f9448ce8b9617fd17d4fb5b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:12.101308 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:12.146782 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e252110e5f9448ce8b9617fd17d4fb5b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.154883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e252110e5f9448ce8b9617fd17d4fb5b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.167648 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e252110e5f9448ce8b9617fd17d4fb5b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.168301 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e252110e5f9448ce8b9617fd17d4fb5b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.175133 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e252110e5f9448ce8b9617fd17d4fb5b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.180957 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1460/2919] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:12 2022] DELETE /identity/v3/users/e252110e5f9448ce8b9617fd17d4fb5b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:12.185523 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:12.185969 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:12.186078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:12.205796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:12.207031 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:12.239314 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:12.266759 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1460/2920] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:12.277077 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:12.278446 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:12.336364 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f145f-106d-4f9a-881a-7306f4425f34 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:12.340639 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:12.341241 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:12.341519 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:12.341787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:12.350839 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:12.352309 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:12.361530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1461/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:12.622972 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:12.624642 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:12.640102 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0f8ca-e9f1-4a92-b8b8-c313eb3b3864 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:12.644994 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:12.645635 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:12.645912 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:12.646184 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] PATH_INFO: `/v3/projects/ad9f9000b995454784def4f7b8c79ee7` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:12.647309 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9f9000b995454784def4f7b8c79ee7)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:12.649328 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:12.695353 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9f9000b995454784def4f7b8c79ee7', 'request_id': 'req-24a30de4-349b-46e9-9df5-0dacd80959ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.701524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9f9000b995454784def4f7b8c79ee7', 'request_id': 'req-24a30de4-349b-46e9-9df5-0dacd80959ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.707527 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9f9000b995454784def4f7b8c79ee7', 'request_id': 'req-24a30de4-349b-46e9-9df5-0dacd80959ce'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.719668 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] The token cache is being invalidate because project ad9f9000b995454784def4f7b8c79ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:12.719984 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.720592 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.722056 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1461/2922] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:12 2022] DELETE /identity/v3/projects/ad9f9000b995454784def4f7b8c79ee7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:12.726043 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:12.735318 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a556c-8d63-4012-aaab-d2ac789e212b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:12.776822 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:12.793093 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:12.793716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:12.793996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:12.794268 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] PATH_INFO: `/v3/users/19d01393202c42d8abe87d46267ebb8e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:12.795373 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d01393202c42d8abe87d46267ebb8e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:12.797331 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:12.842629 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d01393202c42d8abe87d46267ebb8e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.849467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d01393202c42d8abe87d46267ebb8e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.861996 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d01393202c42d8abe87d46267ebb8e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.862645 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d01393202c42d8abe87d46267ebb8e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.869436 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d01393202c42d8abe87d46267ebb8e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:12.875246 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1462/2923] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:12 2022] DELETE /identity/v3/users/19d01393202c42d8abe87d46267ebb8e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:13.147346 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:13.148612 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:13.162071 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a30de4-349b-46e9-9df5-0dacd80959ce admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:13.219396 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:13.220093 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:13.220350 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:13.220617 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] PATH_INFO: `/v3/projects/0c9a542ee6cb4f018398266b178ab97b` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:13.221712 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9a542ee6cb4f018398266b178ab97b)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:13.223596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:13.268078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9a542ee6cb4f018398266b178ab97b', 'request_id': 'req-873b3e0a-05ff-425a-b7e8-744e6373e56f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.273517 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9a542ee6cb4f018398266b178ab97b', 'request_id': 'req-873b3e0a-05ff-425a-b7e8-744e6373e56f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.280143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9a542ee6cb4f018398266b178ab97b', 'request_id': 'req-873b3e0a-05ff-425a-b7e8-744e6373e56f'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.281797 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:13.294563 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] The token cache is being invalidate because project 0c9a542ee6cb4f018398266b178ab97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:13.294848 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.295472 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.296975 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1462/2924] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:13 2022] DELETE /identity/v3/projects/0c9a542ee6cb4f018398266b178ab97b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:13.300917 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:13.302279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:13.315116 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55334121-a78e-4a38-b3f5-22410b644c54 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:13.368810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:13.369922 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:13.370197 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:13.370467 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] PATH_INFO: `/v3/users/9639b8124c2d4282ad3a5a77ad6984c9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:13.371567 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9639b8124c2d4282ad3a5a77ad6984c9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:13.373545 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:13.417608 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9639b8124c2d4282ad3a5a77ad6984c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.424975 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9639b8124c2d4282ad3a5a77ad6984c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.433101 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:13.437878 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9639b8124c2d4282ad3a5a77ad6984c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.438767 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9639b8124c2d4282ad3a5a77ad6984c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.446926 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ae3f0cf2-a769-490a-b4ac-6bde14547892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9639b8124c2d4282ad3a5a77ad6984c9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.453356 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1463/2925] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:13 2022] DELETE /identity/v3/users/9639b8124c2d4282ad3a5a77ad6984c9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:13.718143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:13.719594 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:13.735340 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b3e0a-05ff-425a-b7e8-744e6373e56f admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:13.791173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:13.792361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:13.792637 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:13.792906 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] PATH_INFO: `/v3/projects/18ec64147eae4166bed67abe610885c2` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:13.794002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ec64147eae4166bed67abe610885c2)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:13.795905 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:13.840584 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ec64147eae4166bed67abe610885c2', 'request_id': 'req-ec031490-2c06-447c-840b-3e46890bc619'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.846103 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ec64147eae4166bed67abe610885c2', 'request_id': 'req-ec031490-2c06-447c-840b-3e46890bc619'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.852062 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ec64147eae4166bed67abe610885c2', 'request_id': 'req-ec031490-2c06-447c-840b-3e46890bc619'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.864559 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] The token cache is being invalidate because project 18ec64147eae4166bed67abe610885c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:13.864934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.865540 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-ec031490-2c06-447c-840b-3e46890bc619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:13.867203 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1463/2926] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:13 2022] DELETE /identity/v3/projects/18ec64147eae4166bed67abe610885c2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:13.872536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:13.872826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:13.873106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:13.892450 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:13.893651 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:13.951106 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1464/2927] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:13.959424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:13.959864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:13.960255 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:13.982361 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:13.983562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:13.991415 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1464/2928] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:13.996146 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.005934 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780308fc-eb20-4073-b6e6-81f6c525b873 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.010580 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.011175 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.011453 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.011723 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.012406 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 28 10:00:14.012708 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.014305 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.025495 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1465/2929] 10.222.0.52 () {58 vars in 1153 bytes} [Fri Oct 28 10:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 28 10:00:14.029521 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.038476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e880bed-cb0c-4049-ace9-39dab2050519 None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.043843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.044432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.044710 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.044981 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.045616 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.047489 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.075629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1465/2930] 10.222.0.52 () {60 vars in 1153 bytes} [Fri Oct 28 10:00:14 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:14.079245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.089617 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a9f89-c0ca-4bef-9813-f4d1f318f12c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.145883 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.146407 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.146680 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.146946 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.147564 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.149524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.171099 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1466/2931] 10.222.0.52 () {60 vars in 1144 bytes} [Fri Oct 28 10:00:14 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 28 10:00:14.175306 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.185476 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93790dbd-4d2b-48b2-a7c7-46df7b8b3d47 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.190009 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.190608 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.190883 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.191150 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.191812 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.193425 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.203804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1466/2932] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:14.207502 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.217728 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac9dc4-1cb2-4a7f-bb25-8494adb1679e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.222245 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c161615-e30e-4e2e-845c-87f2562f0644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.222826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c161615-e30e-4e2e-845c-87f2562f0644 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.223096 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c161615-e30e-4e2e-845c-87f2562f0644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.223365 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c161615-e30e-4e2e-845c-87f2562f0644 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.224016 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c161615-e30e-4e2e-845c-87f2562f0644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.225583 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c161615-e30e-4e2e-845c-87f2562f0644 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.236517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1467/2933] 10.222.0.52 () {58 vars in 1122 bytes} [Fri Oct 28 10:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 28 10:00:14.240365 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.250198 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8dd9b-66bc-4dae-8611-3bbb1e808ad5 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.255143 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.255805 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.256084 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.256354 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] PATH_INFO: `/v3/projects/f76f90650ed1421db97994b04e27b21f/users/fbb78c2c48a94056af0b9378ea1c9ab8/roles/39a47ee6b3f14e6ead39370f270c2f38` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.270919 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76f90650ed1421db97994b04e27b21f, user_id=fbb78c2c48a94056af0b9378ea1c9ab8, role_id=39a47ee6b3f14e6ead39370f270c2f38)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.273629 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.286261 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1467/2934] 10.222.0.52 () {60 vars in 1483 bytes} [Fri Oct 28 10:00:14 2022] PUT /identity/v3/projects/f76f90650ed1421db97994b04e27b21f/users/fbb78c2c48a94056af0b9378ea1c9ab8/roles/39a47ee6b3f14e6ead39370f270c2f38 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:14.297164 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.297602 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.297752 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.324787 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] MFA Rules not processed for user `fbb78c2c48a94056af0b9378ea1c9ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:14.325988 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:14.363477 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:14.391087 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1468/2935] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:14.402943 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.404666 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:14.424964 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:14.464089 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977a102-e498-4aa4-a6c2-edc1a953f321 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.467633 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95255a-adc7-4880-bdf1-1bbcdddc92e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '16332ba5fe89453292d21325b333407a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.468257 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95255a-adc7-4880-bdf1-1bbcdddc92e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.468531 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95255a-adc7-4880-bdf1-1bbcdddc92e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.468794 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95255a-adc7-4880-bdf1-1bbcdddc92e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.476804 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95255a-adc7-4880-bdf1-1bbcdddc92e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.478214 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95255a-adc7-4880-bdf1-1bbcdddc92e2 service cinder] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.486868 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1468/2936] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:14.512524 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.572397 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4234ef4-4e52-40b6-b6f5-25d2da96daf3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.577517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.578039 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.578312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.578584 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] PATH_INFO: `/v3/users/fbb78c2c48a94056af0b9378ea1c9ab8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.579682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb78c2c48a94056af0b9378ea1c9ab8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.581616 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.628329 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb78c2c48a94056af0b9378ea1c9ab8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:14.635480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb78c2c48a94056af0b9378ea1c9ab8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:14.649086 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb78c2c48a94056af0b9378ea1c9ab8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:14.649810 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb78c2c48a94056af0b9378ea1c9ab8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:14.657126 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb78c2c48a94056af0b9378ea1c9ab8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:14.662992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1469/2937] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:00:14 2022] DELETE /identity/v3/users/fbb78c2c48a94056af0b9378ea1c9ab8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:14.668130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] REQUEST_METHOD: `POST` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.668419 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.668691 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.687863 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.auth.core [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] MFA Rules not processed for user `66bbceeaa7394d2b922231b2c11b4f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81610) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 28 10:00:14.689054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:14.746400 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1469/2938] 10.222.0.52 () {58 vars in 958 bytes} [Fri Oct 28 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 28 10:00:14.756296 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:14.757682 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:14.815425 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82a3d6-a9b3-4271-a4f0-560af3264b54 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:14.819027 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:14.819624 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:14.819933 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:14.820203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:14.829107 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:14.830536 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:14.835300 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:00:14.840903 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1470/2939] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:00:15.108002 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:00:15.109418 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:00:15.121443 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c84aa9-5fa6-47cb-96e4-47fdbbb66cbc None None] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:00:15.126146 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:00:15.126859 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:00:15.127209 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:00:15.127867 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] PATH_INFO: `/v3/projects/f76f90650ed1421db97994b04e27b21f` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:00:15.128995 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76f90650ed1421db97994b04e27b21f)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:00:15.130887 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:00:15.180562 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76f90650ed1421db97994b04e27b21f', 'request_id': 'req-570f82cc-b04a-429f-abdf-332c8a812e60'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:15.186063 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76f90650ed1421db97994b04e27b21f', 'request_id': 'req-570f82cc-b04a-429f-abdf-332c8a812e60'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:15.192239 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76f90650ed1421db97994b04e27b21f', 'request_id': 'req-570f82cc-b04a-429f-abdf-332c8a812e60'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:15.204702 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] The token cache is being invalidate because project f76f90650ed1421db97994b04e27b21f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81610) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:00:15.204986 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:15.205598 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:00:15.207080 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1470/2940] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:00:15 2022] DELETE /identity/v3/projects/f76f90650ed1421db97994b04e27b21f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:00:25.201779 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:01:30.535102 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:01:30.536589 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:01:30.548144 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677ce8-7806-4337-bc70-dda86decdd0f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:01:30.595759 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:01:30.596370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:01:30.596646 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:01:30.596924 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:01:30.598638 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:01:30.609892 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:01:30.611330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:01:30.619515 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1471/2941] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:01:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:01:40.615731 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=26 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:02:02.963796 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:02:02.965569 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:02.978916 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f82cc-b04a-429f-abdf-332c8a812e60 admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:02:02.982457 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:02:02.983054 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:02:02.983329 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:02:02.983596 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:02:02.985913 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:03.055413 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:02:03.056936 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:02:03.067881 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1471/2942] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:02:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:02:04.258094 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:02:04.269009 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381fe68-0e05-4b21-8203-7b6e09dbc64e service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:02:04.272716 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:02:04.273312 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:02:04.273587 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:02:04.274036 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:02:04.276239 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:04.289880 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:02:04.291299 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:02:04.301921 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1472/2943] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:02:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:02:13.062888 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=34 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:02:14.297114 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:02:16.281386 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:02:16.290231 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4a32a-a696-4b87-bc9a-0ecbb9226e3c service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:02:16.294122 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:02:16.294777 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:02:16.295085 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:02:16.295355 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:02:16.297163 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:16.307778 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:02:16.309218 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:02:16.318293 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1472/2944] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:02:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:02:16.433363 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:02:16.442683 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa50eeb-79bb-4f9d-8d76-db9208db5ff6 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:02:16.446354 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:02:16.447032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:02:16.447211 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:02:16.447480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:02:16.449273 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:16.463462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:02:16.464958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:02:16.475620 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1473/2945] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:02:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:02:16.535149 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:02:16.536524 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:16.603667 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd83de9-d311-4d66-a074-33db0e625fab service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:02:16.608514 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:02:16.609100 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:02:16.609377 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:02:16.609674 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:02:16.611822 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:16.622699 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:02:16.624265 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:02:16.634730 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1473/2946] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:02:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:02:18.754325 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:02:18.755749 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:18.772835 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25371203-9e33-4971-82c5-fd5b4b88f7bb service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:02:18.778370 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:02:18.778966 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:02:18.779243 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:02:18.779517 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:02:18.781254 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:02:18.794149 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:02:18.795736 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:02:18.806771 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1474/2947] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:02:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:02:26.630288 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=38 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:02:28.802158 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=12 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:03:59.365187 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:03:59.369283 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:03:59.452989 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06893c61-7f59-47ad-b95a-ebe71f0d1616 service placement] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:03:59.460498 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dc2bb9976f949548bc4743eb297b082', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:03:59.461213 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:03:59.461568 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:03:59.461966 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:03:59.463768 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:03:59.478746 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:03:59.480909 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:03:59.495727 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1474/2948] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:03:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 28 10:04:02.360480 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:02.425840 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb9698-bee5-43e5-a8dc-91986e5aa5cd service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:02.431893 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:02.432561 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:02.432875 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:02.433151 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:02.434961 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:02.458925 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:02.460541 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:02.470353 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1475/2949] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:04:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:04:04.323640 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:04.342926 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0c086-4560-4a0b-9ccd-b52a80ff46b7 service nova] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:04.346570 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b07b6ea0a9a7475fbcc8d6baf537cc4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:04.347151 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] REQUEST_METHOD: `GET` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:04.347432 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:04.347701 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:04.349436 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:04.435310 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:04.436799 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:04.455811 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1475/2950] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:04:07.427422 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:07.429428 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:07.441910 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b46c66-0f84-4faa-9c4d-b5e029d26117 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:07.447724 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'daf577de783b4e8a9bc9c81dd2a2bd35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6419216789dd4a2f85244acb6bbb9bd8', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:07.448352 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:07.448628 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:07.448902 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:07.450654 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:07.463055 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:07.464826 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:07.476821 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1476/2951] 10.222.0.52 () {60 vars in 1365 bytes} [Fri Oct 28 10:04:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 28 10:04:10.331898 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:10.335072 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:10.352922 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239238e-b91e-488f-be03-5e7c8e87df52 service neutron] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:10.357109 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:10.357745 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:10.358014 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:10.358313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] PATH_INFO: `/v3/users/5b9bb73c501c444ca98c8af9d7b4c7bd` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:10.359447 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b9bb73c501c444ca98c8af9d7b4c7bd)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:10.361399 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:10.413934 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:04:10.416814 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b9bb73c501c444ca98c8af9d7b4c7bd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.425313 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b9bb73c501c444ca98c8af9d7b4c7bd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.438378 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b9bb73c501c444ca98c8af9d7b4c7bd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.439155 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b9bb73c501c444ca98c8af9d7b4c7bd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.451424 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b9bb73c501c444ca98c8af9d7b4c7bd'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.459096 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1476/2952] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:04:10 2022] DELETE /identity/v3/users/5b9bb73c501c444ca98c8af9d7b4c7bd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 28 10:04:10.629573 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:10.645129 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698da576-cebb-474d-9714-f1e119201a3a service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:10.703860 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:10.704418 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:10.704690 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:10.704958 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] PATH_INFO: `/v3/projects/3ce53f0f4dec4042a2cd5a3213b82f5f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:10.706047 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce53f0f4dec4042a2cd5a3213b82f5f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:10.708032 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:10.753203 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce53f0f4dec4042a2cd5a3213b82f5f', 'request_id': 'req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.761052 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce53f0f4dec4042a2cd5a3213b82f5f', 'request_id': 'req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.767281 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce53f0f4dec4042a2cd5a3213b82f5f', 'request_id': 'req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.778046 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] The token cache is being invalidate because project 3ce53f0f4dec4042a2cd5a3213b82f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:04:10.778330 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.778949 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.780462 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1477/2953] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:04:10 2022] DELETE /identity/v3/projects/3ce53f0f4dec4042a2cd5a3213b82f5f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:04:10.784475 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Authenticating user token {{(pid=81610) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:10.785864 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.fernet_utils [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81610) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:10.799130 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bbbbc-3be8-4979-939a-31f31dd35ccc admin admin] Validating token access rules against request {{(pid=81610) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:10.854078 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81610) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:10.854630 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:10.854904 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:10.855173 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] PATH_INFO: `/v3/users/38b9564e0afa4819bb06ea262da591c6` {{(pid=81610) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:10.856324 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b9564e0afa4819bb06ea262da591c6)` {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:10.858238 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] RBAC: Authorization granted {{(pid=81610) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:10.903843 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b9564e0afa4819bb06ea262da591c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.911152 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b9564e0afa4819bb06ea262da591c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.918512 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b9564e0afa4819bb06ea262da591c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.919120 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b9564e0afa4819bb06ea262da591c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.931119 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG keystone.notifications [None req-05815037-e063-4d9f-8a3a-cd9f602ac334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b9564e0afa4819bb06ea262da591c6'} {{(pid=81610) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:10.936956 dv-cinder-zed-3-smb devstack@keystone.service[81610]: [pid: 81610|app: 0|req: 1477/2954] 10.222.0.52 () {58 vars in 1224 bytes} [Fri Oct 28 10:04:10 2022] DELETE /identity/v3/users/38b9564e0afa4819bb06ea262da591c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:04:11.106049 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 28 10:04:11.107530 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 28 10:04:11.120992 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4c21-24fa-4302-aeb3-8f991b7b9fab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 28 10:04:11.124655 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:04:11.182909 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66bbceeaa7394d2b922231b2c11b4f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afd3325807f94fa3b6a7d418f71aaa51', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 28 10:04:11.183542 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 28 10:04:11.183853 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 28 10:04:11.184138 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] PATH_INFO: `/v3/projects/fca3f91c0f054e6f93c94479765057e7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 28 10:04:11.185261 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca3f91c0f054e6f93c94479765057e7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 28 10:04:11.187235 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 28 10:04:11.234396 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca3f91c0f054e6f93c94479765057e7', 'request_id': 'req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:11.243178 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca3f91c0f054e6f93c94479765057e7', 'request_id': 'req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:11.249371 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca3f91c0f054e6f93c94479765057e7', 'request_id': 'req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:11.260873 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] The token cache is being invalidate because project fca3f91c0f054e6f93c94479765057e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 28 10:04:11.261156 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:11.261796 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-573cc0e1-1762-45b5-9d47-f4ff03f4d85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 28 10:04:11.263279 dv-cinder-zed-3-smb devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1478/2955] 10.222.0.52 () {58 vars in 1233 bytes} [Fri Oct 28 10:04:11 2022] DELETE /identity/v3/projects/fca3f91c0f054e6f93c94479765057e7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 28 10:04:20.933537 dv-cinder-zed-3-smb devstack@keystone.service[81610]: DEBUG dbcounter [-] [81610] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81610) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 28 10:04:21.257535 dv-cinder-zed-3-smb devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}